Note: This is test shard 2 of 6.
[==========] Running 7 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 2 tests from DynamicMultiMasterTest
[ RUN ] DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified
2026-08-11T02:57:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:12Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260811 02:57:12.848459 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:35295
--webserver_interface=127.13.94.190
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:35295
--master_support_change_config=false
--master_auto_join_cluster=false with env {}
W20260811 02:57:13.213140 13698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:13.213563 13698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:13.213726 13698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:13.221415 13698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:13.221547 13698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:13.221633 13698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:13.221773 13698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:13.231734 13698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:35295
--master_auto_join_cluster=false
--master_support_change_config=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:35295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:13.233312 13698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:13.235368 13698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:13.249073 13703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:13.249027 13704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:13.250002 13707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:13.250334 13698 server_base.cc:1034] running on GCE node
I20260811 02:57:13.251327 13698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:13.252938 13698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:13.254222 13698 hybrid_clock.cc:648] HybridClock initialized: now 1786417033254184 us; error 83 us; skew 500 ppm
I20260811 02:57:13.254614 13698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:13.257313 13698 webserver.cc:460] Webserver started at http://127.13.94.190:34689/ using document root <none> and password file <none>
I20260811 02:57:13.258157 13698 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:13.258315 13698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:13.258730 13698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:13.261904 13698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/data/instance:
uuid: "c9248e2077724cc9988c3a22a02dab41"
format_stamp: "Formatted at 2026-08-11 02:57:13 on dist-test-slave-p3t5"
I20260811 02:57:13.262780 13698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/wal/instance:
uuid: "c9248e2077724cc9988c3a22a02dab41"
format_stamp: "Formatted at 2026-08-11 02:57:13 on dist-test-slave-p3t5"
I20260811 02:57:13.268497 13698 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 02:57:13.272568 13716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:13.274296 13698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 02:57:13.274524 13698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "c9248e2077724cc9988c3a22a02dab41"
format_stamp: "Formatted at 2026-08-11 02:57:13 on dist-test-slave-p3t5"
I20260811 02:57:13.274768 13698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:13.303707 13698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:13.304711 13698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:13.305297 13698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:13.326418 13698 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:35295
I20260811 02:57:13.326467 13774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:35295 every 8 connection(s)
I20260811 02:57:13.328115 13698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:13.331823 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 13698
I20260811 02:57:13.332149 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-0/wal/instance
I20260811 02:57:13.333702 13775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:13.346807 13775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41: Bootstrap starting.
I20260811 02:57:13.350669 13775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:13.352486 13775 log.cc:824] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:13.356417 13775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41: No bootstrap required, opened a new log
I20260811 02:57:13.366987 13775 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9248e2077724cc9988c3a22a02dab41" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 35295 } }
I20260811 02:57:13.367447 13775 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:13.367622 13775 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9248e2077724cc9988c3a22a02dab41, State: Initialized, Role: FOLLOWER
I20260811 02:57:13.368381 13775 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9248e2077724cc9988c3a22a02dab41" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 35295 } }
I20260811 02:57:13.368692 13775 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:13.368920 13775 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:13.369154 13775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:13.371783 13775 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9248e2077724cc9988c3a22a02dab41" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 35295 } }
I20260811 02:57:13.372505 13775 leader_election.cc:302] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9248e2077724cc9988c3a22a02dab41; no voters:
I20260811 02:57:13.373283 13775 leader_election.cc:288] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:13.373499 13780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:13.374475 13780 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [term 1 LEADER]: Becoming Leader. State: Replica: c9248e2077724cc9988c3a22a02dab41, State: Running, Role: LEADER
I20260811 02:57:13.375370 13780 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9248e2077724cc9988c3a22a02dab41" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 35295 } }
I20260811 02:57:13.376003 13775 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:13.379077 13782 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9248e2077724cc9988c3a22a02dab41. Latest consensus state: current_term: 1 leader_uuid: "c9248e2077724cc9988c3a22a02dab41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9248e2077724cc9988c3a22a02dab41" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 35295 } } }
I20260811 02:57:13.379539 13782 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:13.380241 13781 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9248e2077724cc9988c3a22a02dab41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9248e2077724cc9988c3a22a02dab41" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 35295 } } }
I20260811 02:57:13.380684 13781 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:13.387040 13788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:13.394528 13788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:13.405400 13788 catalog_manager.cc:1347] Generated new cluster ID: 17afebabc6b84274af5a3bf76caabdc4
I20260811 02:57:13.405548 13788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:13.425773 13788 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:13.427258 13788 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:13.439966 13788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9248e2077724cc9988c3a22a02dab41: Generated new TSK 0
I20260811 02:57:13.440965 13788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:13.454077 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:0
--local_ip_for_outbound_sockets=127.13.94.129
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:35295
--builtin_ntp_servers=127.13.94.148:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260811 02:57:13.814304 13802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:13.814628 13802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:13.814795 13802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:13.822345 13802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:13.822605 13802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:57:13.832741 13802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:35295
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:13.834518 13802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:13.836501 13802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:13.851464 13807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:13.851514 13810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:13.851468 13808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:13.852385 13802 server_base.cc:1034] running on GCE node
I20260811 02:57:13.853138 13802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:13.854364 13802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:13.855712 13802 hybrid_clock.cc:648] HybridClock initialized: now 1786417033855639 us; error 44 us; skew 500 ppm
I20260811 02:57:13.856034 13802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:13.858855 13802 webserver.cc:460] Webserver started at http://127.13.94.129:33899/ using document root <none> and password file <none>
I20260811 02:57:13.859606 13802 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:13.859719 13802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:13.860015 13802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:13.863018 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/data/instance:
uuid: "fec1773920fb4f12a3a194d545d4041c"
format_stamp: "Formatted at 2026-08-11 02:57:13 on dist-test-slave-p3t5"
I20260811 02:57:13.863855 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance:
uuid: "fec1773920fb4f12a3a194d545d4041c"
format_stamp: "Formatted at 2026-08-11 02:57:13 on dist-test-slave-p3t5"
I20260811 02:57:13.870265 13802 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 02:57:13.874467 13817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:13.876269 13802 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:13.876519 13802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "fec1773920fb4f12a3a194d545d4041c"
format_stamp: "Formatted at 2026-08-11 02:57:13 on dist-test-slave-p3t5"
I20260811 02:57:13.876776 13802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:13.896729 13802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:13.897828 13802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:13.898191 13802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:13.899351 13802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:13.901087 13802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:13.901172 13802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:13.901350 13802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:13.901414 13802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:13.938683 13802 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:34163
I20260811 02:57:13.938722 13947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:34163 every 8 connection(s)
I20260811 02:57:13.940317 13802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:13.946221 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 13802
I20260811 02:57:13.946439 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:13.956049 13948 heartbeater.cc:347] Connected to a master server at 127.13.94.190:35295
I20260811 02:57:13.956367 13948 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:13.957139 13948 heartbeater.cc:511] Master 127.13.94.190:35295 requested a full tablet report, sending...
I20260811 02:57:13.959333 13735 ts_manager.cc:194] Registered new tserver with Master: fec1773920fb4f12a3a194d545d4041c (127.13.94.129:34163)
I20260811 02:57:13.960916 13735 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.94.129:60461
I20260811 02:57:13.963757 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:13.967689 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:14.964587 13948 heartbeater.cc:503] Master 127.13.94.190:35295 was elected leader, sending a full tablet report...
W20260811 02:57:15.030725 13718 messenger.cc:357] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 6
I20260811 02:57:15.151265 13690 dynamic_multi_master-test.cc:554] Add master stderr: W20260811 02:57:14.331426 13953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:14.331853 13953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:14.332064 13953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:14.340540 13953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:14.340740 13953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:14.340833 13953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:14.340956 13953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:57:14.341082 13953 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20260811 02:57:14.719477 13963 logging.cc:291] disabling asynchronous logging adversely affects Kudu's performance
W20260811 02:57:14.726852 13963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:14.727077 13963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:14.727202 13963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:14.734567 13963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:14.734740 13963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:14.734874 13963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:14.734963 13963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:14.744508 13963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:35295,127.13.94.189:45041
--master_auto_join_cluster=false
--master_support_change_config=false
--master_address_add_new_master=127.13.94.189:45041
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:45041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_async=false
--log_filename=kudu
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:57:14.744949 13963 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:35295,127.13.94.189:45041'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:57:14.746253 13963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:14.748283 13963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:14.761324 13966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:14.761394 13968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:14.761394 13965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:14.762439 13963 server_base.cc:1034] running on GCE node
I20260811 02:57:14.763221 13963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:14.764828 13963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:14.766091 13963 hybrid_clock.cc:648] HybridClock initialized: now 1786417034766026 us; error 33 us; skew 500 ppm
I20260811 02:57:14.766471 13963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:14.768911 13963 webserver.cc:460] Webserver started at http://127.13.94.189:8051/ using document root <none> and password file <none>
I20260811 02:57:14.769695 13963 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:14.769870 13963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:14.770197 13963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:14.780103 13963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/data/instance:
uuid: "322cb8e208f14ee98d5307c141acde60"
format_stamp: "Formatted at 2026-08-11 02:57:14 on dist-test-slave-p3t5"
I20260811 02:57:14.781100 13963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/wal/instance:
uuid: "322cb8e208f14ee98d5307c141acde60"
format_stamp: "Formatted at 2026-08-11 02:57:14 on dist-test-slave-p3t5"
I20260811 02:57:14.788168 13963 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 02:57:14.792253 13975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:14.793810 13963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 02:57:14.793977 13963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "322cb8e208f14ee98d5307c141acde60"
format_stamp: "Formatted at 2026-08-11 02:57:14 on dist-test-slave-p3t5"
I20260811 02:57:14.794162 13963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:14.826123 13963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:14.827105 13963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:14.827458 13963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:14.849601 13963 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:45041
I20260811 02:57:14.849704 14032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:45041 every 8 connection(s)
I20260811 02:57:14.851094 13963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:57:14.855536 14033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:14.862246 14033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 35295 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:14.877324 14033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45041 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:14.895936 14033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60: Bootstrap starting.
I20260811 02:57:14.900815 14033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:14.902338 14033 log.cc:824] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:14.906010 14033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60: No bootstrap required, opened a new log
I20260811 02:57:14.916909 14033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9248e2077724cc9988c3a22a02dab41" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 35295 } } peers { permanent_uuid: "322cb8e208f14ee98d5307c141acde60" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45041 } }
I20260811 02:57:14.917493 14033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:14.917667 14033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 322cb8e208f14ee98d5307c141acde60, State: Initialized, Role: FOLLOWER
I20260811 02:57:14.918596 14033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9248e2077724cc9988c3a22a02dab41" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 35295 } } peers { permanent_uuid: "322cb8e208f14ee98d5307c141acde60" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45041 } }
I20260811 02:57:14.919834 14039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9248e2077724cc9988c3a22a02dab41" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 35295 } } peers { permanent_uuid: "322cb8e208f14ee98d5307c141acde60" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45041 } } }
I20260811 02:57:14.920202 14039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:14.921375 14033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:14.933226 14053 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 322cb8e208f14ee98d5307c141acde60: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:14.933422 14053 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Not implemented: cluster does not support AddMaster with feature(s) DYNAMIC_MULTI_MASTER
I20260811 02:57:15.156049 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 13802
I20260811 02:57:15.165732 13942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:15.249553 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 13802
I20260811 02:57:15.270627 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 13698
I20260811 02:57:15.271934 13770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:15.376989 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 13698
2026-08-11T02:57:15Z chronyd exiting
[ OK ] DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified (2613 ms)
[ RUN ] DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid
2026-08-11T02:57:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:15Z Disabled control of system clock
I20260811 02:57:15.452020 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:42603
--webserver_interface=127.13.94.190
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:42603,127.13.94.189:36955
--master_auto_join_cluster=false with env {}
W20260811 02:57:15.790410 14070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:15.790727 14070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:15.790891 14070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:15.798295 14070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:15.798415 14070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:15.798509 14070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:15.798589 14070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:15.808126 14070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:42603,127.13.94.189:36955
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:42603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:57:15.808568 14070 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:42603,127.13.94.189:36955'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:57:15.809762 14070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:15.811769 14070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:15.824844 14076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:15.824879 14079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:15.824786 14077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:15.825906 14070 server_base.cc:1034] running on GCE node
I20260811 02:57:15.826773 14070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:15.828413 14070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:15.829684 14070 hybrid_clock.cc:648] HybridClock initialized: now 1786417035829607 us; error 60 us; skew 500 ppm
I20260811 02:57:15.830078 14070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:15.832653 14070 webserver.cc:460] Webserver started at http://127.13.94.190:44713/ using document root <none> and password file <none>
I20260811 02:57:15.833393 14070 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:15.833490 14070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:15.833879 14070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:15.836859 14070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/data/instance:
uuid: "cc8c90fdacce4123b14f84bd754fb467"
format_stamp: "Formatted at 2026-08-11 02:57:15 on dist-test-slave-p3t5"
I20260811 02:57:15.837833 14070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/wal/instance:
uuid: "cc8c90fdacce4123b14f84bd754fb467"
format_stamp: "Formatted at 2026-08-11 02:57:15 on dist-test-slave-p3t5"
I20260811 02:57:15.843887 14070 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 02:57:15.847944 14085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:15.849520 14070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:57:15.849786 14070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "cc8c90fdacce4123b14f84bd754fb467"
format_stamp: "Formatted at 2026-08-11 02:57:15 on dist-test-slave-p3t5"
I20260811 02:57:15.850081 14070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:15.865255 14070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:15.866313 14070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:15.866636 14070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:15.885684 14070 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:42603
I20260811 02:57:15.885685 14140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:42603 every 8 connection(s)
I20260811 02:57:15.887308 14070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:15.891531 14141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:15.892761 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 14070
I20260811 02:57:15.892984 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-0/wal/instance
I20260811 02:57:15.894364 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:36955
--webserver_interface=127.13.94.189
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:42603,127.13.94.189:36955
--master_auto_join_cluster=false with env {}
I20260811 02:57:15.898299 14141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:15.916872 14141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } has no permanent_uuid. Determining permanent_uuid...
W20260811 02:57:15.918462 14088 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.94.189:36955: connect: Connection refused (error 111)
W20260811 02:57:15.920137 14141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:36955: Network error: Client connection negotiation failed: client connection to 127.13.94.189:36955: connect: Connection refused (error 111)
I20260811 02:57:15.969641 14141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } attempt: 1
W20260811 02:57:15.971897 14141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:36955: Network error: Client connection negotiation failed: client connection to 127.13.94.189:36955: connect: Connection refused (error 111)
I20260811 02:57:16.040254 14141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } attempt: 2
W20260811 02:57:16.042737 14141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:36955: Network error: Client connection negotiation failed: client connection to 127.13.94.189:36955: connect: Connection refused (error 111)
I20260811 02:57:16.134085 14141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } attempt: 3
W20260811 02:57:16.136396 14141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:36955: Network error: Client connection negotiation failed: client connection to 127.13.94.189:36955: connect: Connection refused (error 111)
W20260811 02:57:16.246155 14145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:16.246487 14145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:16.246567 14145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:16.254117 14145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:16.254244 14145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:16.254287 14145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:16.254347 14145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:16.263870 14145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:42603,127.13.94.189:36955
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:36955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:57:16.264231 14145 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:42603,127.13.94.189:36955'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:57:16.265455 14145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:16.267391 14145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:16.279670 14141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } attempt: 4
W20260811 02:57:16.280078 14154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:16.280259 14155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:16.281348 14158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:16.281699 14145 server_base.cc:1034] running on GCE node
W20260811 02:57:16.281961 14141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:36955: Network error: Client connection negotiation failed: client connection to 127.13.94.189:36955: connect: Connection refused (error 111)
I20260811 02:57:16.282704 14145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:16.284281 14145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:16.285499 14145 hybrid_clock.cc:648] HybridClock initialized: now 1786417036285459 us; error 29 us; skew 500 ppm
I20260811 02:57:16.285895 14145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:16.288504 14145 webserver.cc:460] Webserver started at http://127.13.94.189:42475/ using document root <none> and password file <none>
I20260811 02:57:16.289225 14145 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:16.289319 14145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:16.289671 14145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:16.292660 14145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/data/instance:
uuid: "4e8c5de6f61a4b03832c8678d4262936"
format_stamp: "Formatted at 2026-08-11 02:57:16 on dist-test-slave-p3t5"
I20260811 02:57:16.293542 14145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/wal/instance:
uuid: "4e8c5de6f61a4b03832c8678d4262936"
format_stamp: "Formatted at 2026-08-11 02:57:16 on dist-test-slave-p3t5"
I20260811 02:57:16.299594 14145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260811 02:57:16.303493 14164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:16.305109 14145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:57:16.305276 14145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "4e8c5de6f61a4b03832c8678d4262936"
format_stamp: "Formatted at 2026-08-11 02:57:16 on dist-test-slave-p3t5"
I20260811 02:57:16.305493 14145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:16.321753 14145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:16.322646 14145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:16.322927 14145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:16.342944 14145 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:36955
I20260811 02:57:16.342989 14223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:36955 every 8 connection(s)
I20260811 02:57:16.344450 14145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:57:16.347008 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 14145
I20260811 02:57:16.347221 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/master-1/wal/instance
I20260811 02:57:16.349565 14224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:16.360061 14224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:16.372931 14224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:16.384217 14224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936: Bootstrap starting.
I20260811 02:57:16.388022 14224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:16.389370 14224 log.cc:824] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:16.392297 14224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936: No bootstrap required, opened a new log
I20260811 02:57:16.400136 14224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } }
I20260811 02:57:16.400527 14224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:16.400615 14224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e8c5de6f61a4b03832c8678d4262936, State: Initialized, Role: FOLLOWER
I20260811 02:57:16.401270 14224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } }
I20260811 02:57:16.402220 14230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } } }
I20260811 02:57:16.402616 14230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:16.403359 14224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:16.412784 14241 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:16.412930 14241 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 02:57:16.575127 14172 tablet.cc:2369] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:16.581254 14141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } attempt: 5
I20260811 02:57:16.593029 14141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467: Bootstrap starting.
I20260811 02:57:16.596632 14141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:16.597915 14141 log.cc:824] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:16.600752 14141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467: No bootstrap required, opened a new log
I20260811 02:57:16.608989 14141 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } }
I20260811 02:57:16.610139 14141 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:16.610322 14141 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc8c90fdacce4123b14f84bd754fb467, State: Initialized, Role: FOLLOWER
I20260811 02:57:16.611079 14141 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } }
I20260811 02:57:16.612186 14244 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } } }
I20260811 02:57:16.612680 14244 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:16.613438 14141 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:16.619376 14091 tablet.cc:2369] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:57:16.623769 14255 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:16.623939 14255 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:16.761793 14230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:16.762270 14230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } }
I20260811 02:57:16.764089 14230 leader_election.cc:288] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc8c90fdacce4123b14f84bd754fb467 (127.13.94.190:42603)
I20260811 02:57:16.765435 14115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e8c5de6f61a4b03832c8678d4262936" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc8c90fdacce4123b14f84bd754fb467" is_pre_election: true
I20260811 02:57:16.766196 14115 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e8c5de6f61a4b03832c8678d4262936 in term 0.
I20260811 02:57:16.767143 14166 leader_election.cc:302] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 4e8c5de6f61a4b03832c8678d4262936, cc8c90fdacce4123b14f84bd754fb467; no voters:
I20260811 02:57:16.767845 14230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:16.768009 14230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:16.768121 14230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:16.771498 14230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } }
I20260811 02:57:16.772145 14230 leader_election.cc:288] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [CANDIDATE]: Term 1 election: Requested vote from peers cc8c90fdacce4123b14f84bd754fb467 (127.13.94.190:42603)
I20260811 02:57:16.772872 14115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e8c5de6f61a4b03832c8678d4262936" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc8c90fdacce4123b14f84bd754fb467"
I20260811 02:57:16.773162 14115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:16.776770 14115 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e8c5de6f61a4b03832c8678d4262936 in term 1.
I20260811 02:57:16.777454 14166 leader_election.cc:302] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 4e8c5de6f61a4b03832c8678d4262936, cc8c90fdacce4123b14f84bd754fb467; no voters:
I20260811 02:57:16.777907 14230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:16.778537 14230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [term 1 LEADER]: Becoming Leader. State: Replica: 4e8c5de6f61a4b03832c8678d4262936, State: Running, Role: LEADER
I20260811 02:57:16.779227 14230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } }
I20260811 02:57:16.782894 14257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e8c5de6f61a4b03832c8678d4262936. Latest consensus state: current_term: 1 leader_uuid: "4e8c5de6f61a4b03832c8678d4262936" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } } }
I20260811 02:57:16.783299 14257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:16.784478 14259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:16.786725 14259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:16.796195 14115 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [term 1 FOLLOWER]: Refusing update from remote peer 4e8c5de6f61a4b03832c8678d4262936: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:16.797236 14257 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:16.804366 14244 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e8c5de6f61a4b03832c8678d4262936. Latest consensus state: current_term: 1 leader_uuid: "4e8c5de6f61a4b03832c8678d4262936" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } } }
I20260811 02:57:16.804661 14244 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:16.806900 14230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e8c5de6f61a4b03832c8678d4262936" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } } }
I20260811 02:57:16.807211 14230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:16.807636 14261 mvcc.cc:205] Tried to move back new op lower bound from 7317164182705721344 to 7317164182651990016. Current Snapshot: MvccSnapshot[applied={T|T < 7317164182705721344}]
I20260811 02:57:16.808100 14244 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e8c5de6f61a4b03832c8678d4262936" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8c90fdacce4123b14f84bd754fb467" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 42603 } } peers { permanent_uuid: "4e8c5de6f61a4b03832c8678d4262936" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 36955 } } }
I20260811 02:57:16.808344 14244 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:16.809166 14259 catalog_manager.cc:1347] Generated new cluster ID: 1188c12cb7a54a9aae75fe7991c51c43
I20260811 02:57:16.809332 14259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:16.823612 14259 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:16.824904 14259 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:16.842015 14259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e8c5de6f61a4b03832c8678d4262936: Generated new TSK 0
I20260811 02:57:16.843195 14259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:16.864746 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:0
--local_ip_for_outbound_sockets=127.13.94.129
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:42603,127.13.94.189:36955
--builtin_ntp_servers=127.13.94.148:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260811 02:57:17.230994 14267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:17.231389 14267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:17.231554 14267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:17.239466 14267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:17.239724 14267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:57:17.249410 14267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:42603,127.13.94.189:36955
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:17.250952 14267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:17.253003 14267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:17.268635 14272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:17.268623 14273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:17.268849 14275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:17.269148 14267 server_base.cc:1034] running on GCE node
I20260811 02:57:17.270262 14267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:17.271401 14267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:17.272687 14267 hybrid_clock.cc:648] HybridClock initialized: now 1786417037272587 us; error 71 us; skew 500 ppm
I20260811 02:57:17.273077 14267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:17.276021 14267 webserver.cc:460] Webserver started at http://127.13.94.129:33989/ using document root <none> and password file <none>
I20260811 02:57:17.276791 14267 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:17.276955 14267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:17.277303 14267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:17.280273 14267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/data/instance:
uuid: "c40e1b9dcdee48c3a02e09158f3cf690"
format_stamp: "Formatted at 2026-08-11 02:57:17 on dist-test-slave-p3t5"
I20260811 02:57:17.281145 14267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance:
uuid: "c40e1b9dcdee48c3a02e09158f3cf690"
format_stamp: "Formatted at 2026-08-11 02:57:17 on dist-test-slave-p3t5"
I20260811 02:57:17.287163 14267 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260811 02:57:17.291162 14281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:17.292784 14267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:17.293021 14267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "c40e1b9dcdee48c3a02e09158f3cf690"
format_stamp: "Formatted at 2026-08-11 02:57:17 on dist-test-slave-p3t5"
I20260811 02:57:17.293252 14267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:17.308931 14267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:17.310168 14267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:17.310496 14267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:17.311571 14267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:17.313333 14267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:17.313445 14267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:17.313604 14267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:17.313715 14267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 02:57:17.350013 14267 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:32869
I20260811 02:57:17.350044 14407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:32869 every 8 connection(s)
I20260811 02:57:17.351779 14267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:17.360280 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 14267
I20260811 02:57:17.360560 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:17.383013 14409 heartbeater.cc:347] Connected to a master server at 127.13.94.190:42603
I20260811 02:57:17.383358 14408 heartbeater.cc:347] Connected to a master server at 127.13.94.189:36955
I20260811 02:57:17.383422 14409 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:17.383550 14408 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:17.384434 14408 heartbeater.cc:511] Master 127.13.94.189:36955 requested a full tablet report, sending...
I20260811 02:57:17.384459 14409 heartbeater.cc:511] Master 127.13.94.190:42603 requested a full tablet report, sending...
I20260811 02:57:17.386933 14184 ts_manager.cc:194] Registered new tserver with Master: c40e1b9dcdee48c3a02e09158f3cf690 (127.13.94.129:32869)
I20260811 02:57:17.386951 14102 ts_manager.cc:194] Registered new tserver with Master: c40e1b9dcdee48c3a02e09158f3cf690 (127.13.94.129:32869)
I20260811 02:57:17.388695 14184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.94.129:37985
I20260811 02:57:17.398375 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:17.404300 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
W20260811 02:57:17.413530 14136 generic_service.cc:139] {username='slave'} at 127.0.0.1:49536 forcing change of non-runtime-safe flag leader_failure_max_missed_heartbeat_periods
I20260811 02:57:17.413910 14136 generic_service.cc:175] {username='slave'} at 127.0.0.1:49536 changed flags via RPC: leader_failure_max_missed_heartbeat_periods from '3' to '10.0'
W20260811 02:57:17.415774 14218 generic_service.cc:139] {username='slave'} at 127.0.0.1:44726 forcing change of non-runtime-safe flag leader_failure_max_missed_heartbeat_periods
I20260811 02:57:17.416015 14218 generic_service.cc:175] {username='slave'} at 127.0.0.1:44726 changed flags via RPC: leader_failure_max_missed_heartbeat_periods from '3' to '10.0'
I20260811 02:57:17.416620 13690 test_util.cc:268] Using random seed: -1594910575
I20260811 02:57:17.626014 14255 catalog_manager.cc:1259] Loaded cluster ID: 1188c12cb7a54a9aae75fe7991c51c43
I20260811 02:57:17.626118 14255 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467: loading cluster ID for follower catalog manager: success
I20260811 02:57:17.628800 14255 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467: acquiring CA information for follower catalog manager: success
I20260811 02:57:17.630879 14255 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cc8c90fdacce4123b14f84bd754fb467: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
E20260811 02:57:17.823938 14184 master_service.cc:344] Failed removing master 127.13.94.190:42603. Invalid argument: Mismatch in UUID of the master 127.13.94.190:42603 to be removed. Expected: cc8c90fdacce4123b14f84bd754fb467, supplied: 16790743488716770640.
I20260811 02:57:17.915356 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 14267
I20260811 02:57:17.924927 14403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:18.010411 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 14267
I20260811 02:57:18.032794 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 14070
I20260811 02:57:18.034083 14136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:18.143095 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 14070
I20260811 02:57:18.167573 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 14145
I20260811 02:57:18.168744 14218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:18.276172 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 14145
2026-08-11T02:57:18Z chronyd exiting
[ OK ] DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid (2901 ms)
[----------] 2 tests from DynamicMultiMasterTest (5514 ms total)
[----------] 1 test from AutoAddMasterTest
[ RUN ] AutoAddMasterTest.TestAddWithOnGoingDdl
2026-08-11T02:57:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:18Z Disabled control of system clock
I20260811 02:57:18.349828 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:38889
--webserver_interface=127.13.94.190
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939 with env {}
W20260811 02:57:18.703512 14440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:18.703855 14440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:18.703935 14440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:18.711418 14440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:18.711541 14440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:18.711583 14440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:18.711633 14440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:18.721230 14440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:57:18.721598 14440 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:38889,127.13.94.189:45939'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:57:18.722718 14440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:18.724506 14440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:18.736918 14446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:18.736879 14450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:18.736943 14447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:18.737970 14440 server_base.cc:1034] running on GCE node
I20260811 02:57:18.738816 14440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:18.740360 14440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:18.741585 14440 hybrid_clock.cc:648] HybridClock initialized: now 1786417038741538 us; error 46 us; skew 500 ppm
I20260811 02:57:18.742038 14440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:18.744879 14440 webserver.cc:460] Webserver started at http://127.13.94.190:43213/ using document root <none> and password file <none>
I20260811 02:57:18.745755 14440 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:18.745863 14440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:18.746259 14440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:18.749147 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/instance:
uuid: "92aef555682a43a1a1619b2bc9ba7ce1"
format_stamp: "Formatted at 2026-08-11 02:57:18 on dist-test-slave-p3t5"
I20260811 02:57:18.750051 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal/instance:
uuid: "92aef555682a43a1a1619b2bc9ba7ce1"
format_stamp: "Formatted at 2026-08-11 02:57:18 on dist-test-slave-p3t5"
I20260811 02:57:18.755761 14440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 02:57:18.759506 14456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:18.761066 14440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:57:18.761274 14440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "92aef555682a43a1a1619b2bc9ba7ce1"
format_stamp: "Formatted at 2026-08-11 02:57:18 on dist-test-slave-p3t5"
I20260811 02:57:18.761500 14440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:18.780169 14440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:18.781250 14440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:18.781584 14440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:18.800949 14440 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:38889
I20260811 02:57:18.800994 14511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:38889 every 8 connection(s)
I20260811 02:57:18.802604 14440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:18.806829 14513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:18.812179 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 14440
I20260811 02:57:18.812431 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal/instance
I20260811 02:57:18.813766 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:45939
--webserver_interface=127.13.94.189
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939 with env {}
I20260811 02:57:18.813364 14513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:18.832809 14513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } has no permanent_uuid. Determining permanent_uuid...
W20260811 02:57:18.834345 14457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.94.189:45939: connect: Connection refused (error 111)
W20260811 02:57:18.836128 14513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:45939: Network error: Client connection negotiation failed: client connection to 127.13.94.189:45939: connect: Connection refused (error 111)
I20260811 02:57:18.885583 14513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } attempt: 1
W20260811 02:57:18.887887 14513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:45939: Network error: Client connection negotiation failed: client connection to 127.13.94.189:45939: connect: Connection refused (error 111)
I20260811 02:57:18.956300 14513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } attempt: 2
W20260811 02:57:18.958978 14513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:45939: Network error: Client connection negotiation failed: client connection to 127.13.94.189:45939: connect: Connection refused (error 111)
I20260811 02:57:19.050340 14513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } attempt: 3
W20260811 02:57:19.052591 14513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:45939: Network error: Client connection negotiation failed: client connection to 127.13.94.189:45939: connect: Connection refused (error 111)
W20260811 02:57:19.165696 14517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:19.166081 14517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:19.166163 14517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:19.173785 14517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:19.173949 14517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:19.174052 14517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:19.174160 14517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:19.183826 14517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:45939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:57:19.184298 14517 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:38889,127.13.94.189:45939'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:57:19.185518 14517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:19.187512 14517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:19.195926 14513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } attempt: 4
W20260811 02:57:19.198145 14513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:45939: Network error: Client connection negotiation failed: client connection to 127.13.94.189:45939: connect: Connection refused (error 111)
W20260811 02:57:19.200280 14526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:19.200531 14528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:19.200781 14524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:19.201658 14517 server_base.cc:1034] running on GCE node
I20260811 02:57:19.202628 14517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:19.204201 14517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:19.205421 14517 hybrid_clock.cc:648] HybridClock initialized: now 1786417039205368 us; error 39 us; skew 500 ppm
I20260811 02:57:19.205866 14517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:19.208531 14517 webserver.cc:460] Webserver started at http://127.13.94.189:39639/ using document root <none> and password file <none>
I20260811 02:57:19.209323 14517 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:19.209451 14517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:19.209895 14517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:19.212832 14517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/instance:
uuid: "d72eb236aa6043eb91a5dfce161d5057"
format_stamp: "Formatted at 2026-08-11 02:57:19 on dist-test-slave-p3t5"
I20260811 02:57:19.213810 14517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal/instance:
uuid: "d72eb236aa6043eb91a5dfce161d5057"
format_stamp: "Formatted at 2026-08-11 02:57:19 on dist-test-slave-p3t5"
I20260811 02:57:19.219774 14517 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 02:57:19.223879 14534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:19.225445 14517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 02:57:19.225726 14517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "d72eb236aa6043eb91a5dfce161d5057"
format_stamp: "Formatted at 2026-08-11 02:57:19 on dist-test-slave-p3t5"
I20260811 02:57:19.225981 14517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:19.248029 14517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:19.249032 14517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:19.249363 14517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:19.269264 14517 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:45939
I20260811 02:57:19.269287 14596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:45939 every 8 connection(s)
I20260811 02:57:19.270838 14517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:57:19.275058 14597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:19.277341 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 14517
I20260811 02:57:19.277556 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal/instance
I20260811 02:57:19.282868 14597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:19.297890 14597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:19.308746 14597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Bootstrap starting.
I20260811 02:57:19.312369 14597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:19.313777 14597 log.cc:824] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:19.316576 14597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: No bootstrap required, opened a new log
I20260811 02:57:19.324414 14597 raft_consensus.cc:348] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } }
I20260811 02:57:19.324859 14597 raft_consensus.cc:374] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:19.325009 14597 raft_consensus.cc:729] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72eb236aa6043eb91a5dfce161d5057, State: Initialized, Role: FOLLOWER
I20260811 02:57:19.325866 14597 consensus_queue.cc:260] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } }
I20260811 02:57:19.326952 14604 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } }
I20260811 02:57:19.327327 14604 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:19.328063 14597 sys_catalog.cc:564] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:19.339250 14616 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:19.339419 14616 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:19.335594 14517 master_runner.cc:186] Error getting master registration for 127.13.94.190:38889: OK, instance_id { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" instance_seqno: 1786417038795519 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:57:19.343236 14517 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:57:19.497493 14513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } attempt: 5
W20260811 02:57:19.501821 14541 tablet.cc:2369] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:19.510610 14513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Bootstrap starting.
I20260811 02:57:19.514889 14513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:19.516330 14513 log.cc:824] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:19.519834 14513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: No bootstrap required, opened a new log
I20260811 02:57:19.531502 14513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } }
I20260811 02:57:19.531989 14513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:19.532102 14513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92aef555682a43a1a1619b2bc9ba7ce1, State: Initialized, Role: FOLLOWER
I20260811 02:57:19.532855 14513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } }
I20260811 02:57:19.533908 14620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } }
I20260811 02:57:19.534431 14620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:19.535290 14513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:19.545861 14632 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:19.546213 14632 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:19.549105 14440 master_runner.cc:263] No leader master found from master 92aef555682a43a1a1619b2bc9ba7ce1
I20260811 02:57:19.549268 14440 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:57:19.648108 14620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:19.648514 14620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } }
I20260811 02:57:19.650249 14620 leader_election.cc:288] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d72eb236aa6043eb91a5dfce161d5057 (127.13.94.189:45939)
I20260811 02:57:19.651203 14567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72eb236aa6043eb91a5dfce161d5057" is_pre_election: true
I20260811 02:57:19.651942 14567 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 0.
I20260811 02:57:19.652669 14457 leader_election.cc:302] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 92aef555682a43a1a1619b2bc9ba7ce1, d72eb236aa6043eb91a5dfce161d5057; no voters:
I20260811 02:57:19.653157 14620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:19.653335 14620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:19.653442 14620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:19.656874 14620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } }
I20260811 02:57:19.657490 14620 leader_election.cc:288] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 1 election: Requested vote from peers d72eb236aa6043eb91a5dfce161d5057 (127.13.94.189:45939)
I20260811 02:57:19.658203 14567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72eb236aa6043eb91a5dfce161d5057"
I20260811 02:57:19.658461 14567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:19.661901 14567 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 1.
I20260811 02:57:19.662452 14457 leader_election.cc:302] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 92aef555682a43a1a1619b2bc9ba7ce1, d72eb236aa6043eb91a5dfce161d5057; no voters:
I20260811 02:57:19.662853 14620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:19.663457 14620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 LEADER]: Becoming Leader. State: Replica: 92aef555682a43a1a1619b2bc9ba7ce1, State: Running, Role: LEADER
I20260811 02:57:19.664152 14620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } }
I20260811 02:57:19.667969 14634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } }
I20260811 02:57:19.668418 14634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:19.669305 14636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:19.671583 14636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:19.683733 14567 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 1 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:19.684871 14620 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:19.694933 14604 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } }
I20260811 02:57:19.695240 14604 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:19.710302 14604 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } }
I20260811 02:57:19.710628 14604 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:19.712037 14620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } }
I20260811 02:57:19.712311 14620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:19.713858 14636 catalog_manager.cc:1347] Generated new cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:19.713963 14636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:19.736071 14636 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:19.738240 14636 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:19.751784 14636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Generated new TSK 0
I20260811 02:57:19.752991 14636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:19.779573 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:0
--local_ip_for_outbound_sockets=127.13.94.129
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260811 02:57:20.149765 14643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:20.150099 14643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:20.150247 14643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:20.157809 14643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:20.158066 14643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:57:20.167827 14643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:20.169310 14643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:20.171295 14643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:20.185717 14652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:20.185773 14649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:20.185688 14648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:20.186596 14643 server_base.cc:1034] running on GCE node
I20260811 02:57:20.187295 14643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:20.188299 14643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:20.189544 14643 hybrid_clock.cc:648] HybridClock initialized: now 1786417040189473 us; error 38 us; skew 500 ppm
I20260811 02:57:20.189926 14643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:20.192344 14643 webserver.cc:460] Webserver started at http://127.13.94.129:34685/ using document root <none> and password file <none>
I20260811 02:57:20.193076 14643 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:20.193230 14643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:20.193579 14643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:20.196967 14643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/instance:
uuid: "42a22db0f41a4489a0dc8ce5315e4428"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:20.197911 14643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance:
uuid: "42a22db0f41a4489a0dc8ce5315e4428"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:20.203557 14643 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 02:57:20.207350 14660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:20.209008 14643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:57:20.209249 14643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "42a22db0f41a4489a0dc8ce5315e4428"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:20.209491 14643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:20.227087 14643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:20.228057 14643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:20.228430 14643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:20.229488 14643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:20.231295 14643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:20.231375 14643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:20.231549 14643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:20.231613 14643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:20.267946 14643 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:37837
I20260811 02:57:20.267987 14785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:37837 every 8 connection(s)
I20260811 02:57:20.269776 14643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:20.274991 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 14643
I20260811 02:57:20.275240 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:20.300858 14787 heartbeater.cc:347] Connected to a master server at 127.13.94.190:38889
I20260811 02:57:20.301292 14787 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:20.301573 14786 heartbeater.cc:347] Connected to a master server at 127.13.94.189:45939
I20260811 02:57:20.301811 14786 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:20.302423 14786 heartbeater.cc:511] Master 127.13.94.189:45939 requested a full tablet report, sending...
I20260811 02:57:20.302443 14787 heartbeater.cc:511] Master 127.13.94.190:38889 requested a full tablet report, sending...
I20260811 02:57:20.304493 14474 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:20.304608 14556 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:20.306295 14474 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.94.129:33121
I20260811 02:57:20.313997 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:20.319574 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:20.319885 13690 test_util.cc:268] Using random seed: -1592007300
I20260811 02:57:20.341961 14616 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:20.342063 14616 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: loading cluster ID for follower catalog manager: success
I20260811 02:57:20.344837 14616 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: acquiring CA information for follower catalog manager: success
I20260811 02:57:20.347040 14616 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:20.351886 14474 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53476:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:20.380849 14716 tablet_service.cc:1467] Processing CreateTablet for tablet f78e24d030f549fb86183de598e3576b (DEFAULT_TABLE table=test-workload [id=f0728c3f7ecc49b5ae49a5bd89ce2df8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:20.382249 14716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78e24d030f549fb86183de598e3576b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:20.394011 14809 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:20.397663 14809 tablet_bootstrap.cc:654] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:20.398830 14809 log.cc:824] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:20.401629 14809 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:20.402043 14809 ts_tablet_manager.cc:1397] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260811 02:57:20.409577 14809 raft_consensus.cc:348] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.410123 14809 raft_consensus.cc:374] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:20.410239 14809 raft_consensus.cc:729] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:20.410904 14809 consensus_queue.cc:260] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.411252 14809 raft_consensus.cc:388] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:20.411424 14809 raft_consensus.cc:482] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:20.411638 14809 raft_consensus.cc:3037] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:20.413949 14809 raft_consensus.cc:504] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.414438 14809 leader_election.cc:302] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:20.415124 14809 leader_election.cc:288] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:20.415299 14812 raft_consensus.cc:2781] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:20.416040 14812 raft_consensus.cc:686] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:20.416337 14787 heartbeater.cc:503] Master 127.13.94.190:38889 was elected leader, sending a full tablet report...
I20260811 02:57:20.416710 14812 consensus_queue.cc:237] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.417148 14809 ts_tablet_manager.cc:1428] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260811 02:57:20.422897 14474 catalog_manager.cc:5495] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:20.441835 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:44361
--webserver_interface=127.13.94.188
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361 with env {}
I20260811 02:57:20.472281 14474 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53492:
name: "default.0_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:20.490142 14716 tablet_service.cc:1467] Processing CreateTablet for tablet a4ff2780d1b54a829f5a65f9b36d10ec (DEFAULT_TABLE table=default.0_1 [id=6c64c679309b4f29ab1ba582a3bcff07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:20.490257 14473 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53508:
name: "default.1_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:20.490746 14716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ff2780d1b54a829f5a65f9b36d10ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:20.499498 14809 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:20.501410 14809 tablet_bootstrap.cc:654] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:20.504753 14809 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:20.505301 14809 ts_tablet_manager.cc:1397] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:57:20.506445 14809 raft_consensus.cc:348] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.506739 14809 raft_consensus.cc:374] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:20.506898 14809 raft_consensus.cc:729] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:20.507211 14809 consensus_queue.cc:260] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.507375 14716 tablet_service.cc:1467] Processing CreateTablet for tablet ee6c637392954d28abe1172c900498b2 (DEFAULT_TABLE table=default.1_1 [id=2b6d1a80a21d49ec8120aeac1c9b2764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:20.507576 14809 raft_consensus.cc:388] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:20.507954 14809 raft_consensus.cc:482] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:20.508109 14809 raft_consensus.cc:3037] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:20.507850 14716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee6c637392954d28abe1172c900498b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:20.510377 14809 raft_consensus.cc:504] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.510778 14809 leader_election.cc:302] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:20.511122 14812 raft_consensus.cc:2781] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:20.511293 14812 raft_consensus.cc:686] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:20.511550 14812 consensus_queue.cc:237] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.517871 14809 leader_election.cc:288] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:20.519979 14809 ts_tablet_manager.cc:1428] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.014s user 0.002s sys 0.003s
I20260811 02:57:20.520062 14473 catalog_manager.cc:5495] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:20.520319 14809 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:20.522280 14809 tablet_bootstrap.cc:654] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:20.524544 14809 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:20.524760 14809 ts_tablet_manager.cc:1397] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:20.526299 14809 raft_consensus.cc:348] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.526584 14809 raft_consensus.cc:374] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:20.526702 14809 raft_consensus.cc:729] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:20.527000 14809 consensus_queue.cc:260] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.528656 14809 raft_consensus.cc:388] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:20.528784 14809 raft_consensus.cc:482] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:20.528947 14809 raft_consensus.cc:3037] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:20.532029 14809 raft_consensus.cc:504] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.532358 14809 leader_election.cc:302] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:20.532660 14812 raft_consensus.cc:2781] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:20.532804 14812 raft_consensus.cc:686] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:20.533030 14812 consensus_queue.cc:237] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:20.534092 14809 leader_election.cc:288] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:20.534904 14809 ts_tablet_manager.cc:1428] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260811 02:57:20.540947 14474 catalog_manager.cc:5495] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
W20260811 02:57:20.854720 14827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:20.855038 14827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:20.855194 14827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:20.862694 14827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:20.862816 14827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:20.862936 14827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:20.863037 14827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:20.873782 14827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:44361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.94.188
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:20.875326 14827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:20.877489 14827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:20.891240 14844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:20.891549 14843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:20.891345 14846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:20.892645 14827 server_base.cc:1034] running on GCE node
I20260811 02:57:20.893822 14827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:20.895503 14827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:20.896811 14827 hybrid_clock.cc:648] HybridClock initialized: now 1786417040896751 us; error 40 us; skew 500 ppm
I20260811 02:57:20.897240 14827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:20.900084 14827 webserver.cc:460] Webserver started at http://127.13.94.188:36707/ using document root <none> and password file <none>
I20260811 02:57:20.900883 14827 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:20.901039 14827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:20.901392 14827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:20.904294 14827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/instance:
uuid: "e3aa822be3af4af4a40b453c83d7d425"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:20.905138 14827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal/instance:
uuid: "e3aa822be3af4af4a40b453c83d7d425"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:20.911092 14827 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 02:57:20.915103 14852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:20.916736 14827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:20.916968 14827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
uuid: "e3aa822be3af4af4a40b453c83d7d425"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:20.917207 14827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:20.935488 14827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:20.936619 14827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:20.936991 14827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:20.960177 14827 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.188:44361
I20260811 02:57:20.960220 14913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.188:44361 every 8 connection(s)
I20260811 02:57:20.961845 14827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
I20260811 02:57:20.965951 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 14827
I20260811 02:57:20.966645 14914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:20.976848 14914 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:20.989392 14914 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:20.996696 14914 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:21.008236 14914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap starting.
I20260811 02:57:21.011974 14914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:21.013136 14914 log.cc:824] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:21.016042 14914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: No bootstrap required, opened a new log
I20260811 02:57:21.023910 14914 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } }
I20260811 02:57:21.024324 14914 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:21.024485 14914 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3aa822be3af4af4a40b453c83d7d425, State: Initialized, Role: FOLLOWER
I20260811 02:57:21.025184 14914 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } }
I20260811 02:57:21.026252 14921 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } }
I20260811 02:57:21.026582 14921 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:21.027420 14914 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:21.038887 14932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:21.039218 14932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:21.043452 14827 master_runner.cc:428] Detected that this master e3aa822be3af4af4a40b453c83d7d425 is joining an existing cluster
I20260811 02:57:21.043599 14827 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.94.188:44361
I20260811 02:57:21.085700 14473 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.94.188:44361: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425"
member_type: NON_VOTER
last_known_addr {
host: "127.13.94.188"
port: 44361
}
attrs {
promote: true
}
}
dest_uuid: "92aef555682a43a1a1619b2bc9ba7ce1"
cas_config_opid_index: -1
I20260811 02:57:21.087204 14473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } }
I20260811 02:57:21.090436 14567 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 1 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20260811 02:57:21.090664 14886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:21.091156 14944 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260811 02:57:21.094698 14886 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20260811 02:57:21.094285 14944 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index -1 to 15, NON_VOTER e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.094962 14567 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index -1 to 15, NON_VOTER e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.098145 14943 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:21.105724 14944 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.94.188:44361
I20260811 02:57:21.107091 14827 master.cc:537] Master@127.13.94.188:44361 shutting down...
I20260811 02:57:21.109836 14944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.110380 14944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:21.110906 14946 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.111352 14946 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 02:57:21.114673 14473 master.cc:648] Remote error: unable to get registration information for peer e3aa822be3af4af4a40b453c83d7d425: Service unavailable: service kudu.master.MasterService not registered on Master
I20260811 02:57:21.118389 14945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.118950 14945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:21.148075 14886 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 LEARNER]: Committing config change with OpId 1.15: config changed from index -1 to 15, NON_VOTER e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
W20260811 02:57:21.155155 14473 master.cc:648] Remote error: unable to get registration information for peer e3aa822be3af4af4a40b453c83d7d425: Service unavailable: service kudu.master.MasterService not registered on Master
I20260811 02:57:21.154896 14947 mvcc.cc:205] Tried to move back new op lower bound from 7317164198061416448 to 7317164194469126144. Current Snapshot: MvccSnapshot[applied={T|T < 7317164197579657216 or (T in {7317164197784653824,7317164197818363904,7317164197858365440,7317164197887139840,7317164197978775552,7317164198014771200})}]
I20260811 02:57:21.156564 14955 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.156967 14955 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:21.157033 14921 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.157346 14921 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:21.157761 14950 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.158193 14950 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:21.160542 14827 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 LEARNER]: Raft consensus shutting down.
I20260811 02:57:21.161129 14827 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 LEARNER]: Raft consensus is shut down!
I20260811 02:57:21.161320 14827 tablet_replica.cc:330] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: stopping tablet replica
I20260811 02:57:21.178848 14827 master.cc:552] Master@127.13.94.188:44361 shutdown complete.
I20260811 02:57:21.179015 14827 master_runner.cc:305] Clearing existing system tablet
I20260811 02:57:21.179483 14827 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:21.179606 14827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:21.182806 14827 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.003s
I20260811 02:57:21.184576 14957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:21.185068 14827 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:21.185372 14827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:21.185503 14827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
uuid: "e3aa822be3af4af4a40b453c83d7d425"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:21.188947 14827 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:21.194053 14827 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 02:57:21.194242 14827 log.cc:1196] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Deleting WAL directory at /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 02:57:21.194797 14827 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Deleting consensus metadata
I20260811 02:57:21.195071 14827 master_runner.cc:315] Copying system tablet from 127.13.94.190:38889
I20260811 02:57:21.198340 14827 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: tablet copy: Beginning tablet copy session from remote peer at address 127.13.94.190:38889
I20260811 02:57:21.206965 14495 tablet_copy_service.cc:140] P 92aef555682a43a1a1619b2bc9ba7ce1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e3aa822be3af4af4a40b453c83d7d425 ({username='slave'} at 127.0.0.1:53534)
I20260811 02:57:21.207370 14495 tablet_copy_service.cc:161] P 92aef555682a43a1a1619b2bc9ba7ce1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e3aa822be3af4af4a40b453c83d7d425 at {username='slave'} at 127.0.0.1:53534: session id = e3aa822be3af4af4a40b453c83d7d425-00000000000000000000000000000000
I20260811 02:57:21.211048 14495 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 02:57:21.213606 14827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 02:57:21.218758 14827 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: tablet copy: Starting download of 0 data blocks...
I20260811 02:57:21.219161 14827 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: tablet copy: Starting download of 1 WAL segments...
I20260811 02:57:21.223097 14827 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260811 02:57:21.225723 14473 master.cc:648] Network error: unable to get registration information for peer e3aa822be3af4af4a40b453c83d7d425: Client connection negotiation failed: client connection to 127.13.94.188:44361: connect: Connection refused (error 111)
I20260811 02:57:21.227999 14495 tablet_copy_service.cc:342] P 92aef555682a43a1a1619b2bc9ba7ce1: Request end of tablet copy session e3aa822be3af4af4a40b453c83d7d425-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:53534
I20260811 02:57:21.228363 14495 tablet_copy_service.cc:433] P 92aef555682a43a1a1619b2bc9ba7ce1: ending tablet copy session e3aa822be3af4af4a40b453c83d7d425-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e3aa822be3af4af4a40b453c83d7d425
I20260811 02:57:21.240983 14827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:21.243988 14966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:21.244545 14965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:21.245144 14968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:21.251500 14827 server_base.cc:1034] running on GCE node
I20260811 02:57:21.251909 14827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:21.252645 14827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:21.253850 14827 hybrid_clock.cc:648] HybridClock initialized: now 1786417041253794 us; error 46 us; skew 500 ppm
I20260811 02:57:21.254140 14827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:21.255820 14827 webserver.cc:460] Webserver started at http://127.13.94.188:33453/ using document root <none> and password file <none>
I20260811 02:57:21.256206 14827 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:21.256294 14827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:21.259625 14827 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 02:57:21.261720 14975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:21.262204 14827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 02:57:21.262360 14827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
uuid: "e3aa822be3af4af4a40b453c83d7d425"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:21.262524 14827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:21.284426 14827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:21.285385 14827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:21.313450 15035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.188:44361 every 8 connection(s)
I20260811 02:57:21.313514 14827 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.188:44361
I20260811 02:57:21.314801 14827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
I20260811 02:57:21.319025 15036 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:21.321776 15036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap starting.
I20260811 02:57:21.342743 15036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=9 ignored=0} mutations{seen=7 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:21.343362 15036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap complete.
I20260811 02:57:21.344453 15036 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } }
I20260811 02:57:21.344769 15036 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e3aa822be3af4af4a40b453c83d7d425, State: Initialized, Role: LEARNER
I20260811 02:57:21.345039 15036 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } }
I20260811 02:57:21.345956 15039 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.346318 15039 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:21.346395 15036 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:21.356088 15051 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:21.356240 15051 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: loading cluster ID for follower catalog manager: success
I20260811 02:57:21.359804 15051 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: acquiring CA information for follower catalog manager: success
I20260811 02:57:21.363719 15051 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:21.557588 14473 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53536:
name: "default.0_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:21.564141 14944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.564502 14944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:21.571434 14474 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53540:
name: "default.1_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:21.579030 15039 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:21.579457 15039 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:21.580379 14944 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: attempting to promote NON_VOTER e3aa822be3af4af4a40b453c83d7d425 to VOTER
I20260811 02:57:21.580996 14944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 1.17, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:21.588037 14564 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 1 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20260811 02:57:21.589133 15069 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260811 02:57:21.593410 14945 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 LEADER]: Committing config change with OpId 1.18: config changed from index 15 to 18, e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:21.594328 14564 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 15 to 18, e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:21.597002 15006 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 LEARNER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20260811 02:57:21.598083 14943 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260811 02:57:21.599345 14944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:21.599639 14944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:21.599792 14943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:21.600042 14943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:21.600310 15006 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 15 to 18, e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:21.601135 14946 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:21.601382 14946 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:21.605547 15039 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:21.605836 15039 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:21.606211 14716 tablet_service.cc:1467] Processing CreateTablet for tablet 11ebf0753b444c0fb704488c06204383 (DEFAULT_TABLE table=default.0_2 [id=5bf294e9599741cab356594a0889edc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:21.606715 14716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ebf0753b444c0fb704488c06204383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:21.606773 14943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:21.607048 14943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:21.611810 14714 tablet_service.cc:1467] Processing CreateTablet for tablet e55d01af74a84dc391101460f2a9c063 (DEFAULT_TABLE table=default.1_2 [id=acb767fa16b7453da75b503e3c4be4f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:21.612272 14714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e55d01af74a84dc391101460f2a9c063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:21.615265 15077 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:21.617461 15077 tablet_bootstrap.cc:654] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:21.620613 15077 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:21.620805 15077 ts_tablet_manager.cc:1397] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:57:21.621506 15077 raft_consensus.cc:348] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:21.621780 15077 raft_consensus.cc:374] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:21.621881 15077 raft_consensus.cc:729] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:21.622102 15077 consensus_queue.cc:260] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:21.622306 15077 raft_consensus.cc:388] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:21.622404 15077 raft_consensus.cc:482] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:21.622489 15077 raft_consensus.cc:3037] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:21.624696 15077 raft_consensus.cc:504] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:21.625010 15077 leader_election.cc:302] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:21.625561 15077 leader_election.cc:288] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:21.625751 15079 raft_consensus.cc:2781] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:21.626345 15079 raft_consensus.cc:686] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:21.626695 15079 consensus_queue.cc:237] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:21.629410 14474 catalog_manager.cc:5495] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:21.635097 15077 ts_tablet_manager.cc:1428] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.014s user 0.008s sys 0.003s
I20260811 02:57:21.637473 15077 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:21.640482 15077 tablet_bootstrap.cc:654] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:21.642990 15077 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:21.643201 15077 ts_tablet_manager.cc:1397] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 02:57:21.646173 15077 raft_consensus.cc:348] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:21.646492 15077 raft_consensus.cc:374] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:21.646588 15077 raft_consensus.cc:729] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:21.646883 15077 consensus_queue.cc:260] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:21.647102 15077 raft_consensus.cc:388] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:21.647212 15077 raft_consensus.cc:482] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:21.647328 15077 raft_consensus.cc:3037] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:21.650137 15077 raft_consensus.cc:504] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:21.650454 15077 leader_election.cc:302] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:21.650699 15077 leader_election.cc:288] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:21.650755 15079 raft_consensus.cc:2781] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:21.651077 15079 raft_consensus.cc:686] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:21.651238 15077 ts_tablet_manager.cc:1428] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
I20260811 02:57:21.651331 15079 consensus_queue.cc:237] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:21.653856 14474 catalog_manager.cc:5495] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:21.699595 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 14643
I20260811 02:57:21.707800 14781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:21.804477 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 14643
I20260811 02:57:21.828487 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 14440
I20260811 02:57:21.829941 14507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:21.946388 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 14440
I20260811 02:57:21.973773 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 14517
I20260811 02:57:21.975008 14592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:22.093575 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 14517
I20260811 02:57:22.119261 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 14827
I20260811 02:57:22.120404 15030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:22.234193 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 14827
I20260811 02:57:22.260438 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:38889
--webserver_interface=127.13.94.190
--webserver_port=43213
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361 with env {}
W20260811 02:57:22.611114 15094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:22.611452 15094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:22.611533 15094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:22.618988 15094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:22.619124 15094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:22.619168 15094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:22.619230 15094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:22.628919 15094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=43213
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:22.630326 15094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:22.632203 15094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:22.645071 15100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:22.645161 15102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:22.645138 15099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:22.646234 15094 server_base.cc:1034] running on GCE node
I20260811 02:57:22.647204 15094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:22.649984 15094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:22.651289 15094 hybrid_clock.cc:648] HybridClock initialized: now 1786417042651171 us; error 96 us; skew 500 ppm
I20260811 02:57:22.651681 15094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:22.665942 15094 webserver.cc:460] Webserver started at http://127.13.94.190:43213/ using document root <none> and password file <none>
I20260811 02:57:22.667084 15094 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:22.667281 15094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:22.680087 15094 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.003s sys 0.002s
I20260811 02:57:22.684707 15125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:22.686722 15094 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 02:57:22.686991 15094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "92aef555682a43a1a1619b2bc9ba7ce1"
format_stamp: "Formatted at 2026-08-11 02:57:18 on dist-test-slave-p3t5"
I20260811 02:57:22.688481 15094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:22.720834 15094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:22.721882 15094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:22.722216 15094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:22.746922 15094 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:38889
I20260811 02:57:22.746987 15183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:38889 every 8 connection(s)
I20260811 02:57:22.748656 15094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:22.753396 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 15094
I20260811 02:57:22.754215 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:45939
--webserver_interface=127.13.94.189
--webserver_port=39639
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361 with env {}
I20260811 02:57:22.765607 15184 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:22.772464 15184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Bootstrap starting.
I20260811 02:57:22.806583 15184 log.cc:824] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:22.839478 15184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=0 applied=23 ignored=0} inserts{seen=13 ignored=0} mutations{seen=12 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:22.840356 15184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Bootstrap complete.
I20260811 02:57:22.853049 15184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:22.854436 15184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92aef555682a43a1a1619b2bc9ba7ce1, State: Initialized, Role: FOLLOWER
I20260811 02:57:22.855222 15184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 1.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:22.856236 15192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:22.856619 15192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:22.857702 15184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:22.880434 15205 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:22.880596 15205 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: loading cluster ID for follower catalog manager: success
I20260811 02:57:22.885265 15205 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: acquiring CA information for follower catalog manager: success
I20260811 02:57:22.888139 15205 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:22.893240 15094 master_runner.cc:186] Error getting master registration for 127.13.94.189:45939: Network error: Client connection negotiation failed: client connection to 127.13.94.189:45939: connect: Connection refused (error 111),
I20260811 02:57:22.899817 15094 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:57:23.273576 15187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:23.274124 15187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:23.274267 15187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:23.286620 15187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:23.286816 15187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:23.286899 15187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:23.286999 15187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:23.302767 15187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:45939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--webserver_port=39639
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:23.304698 15187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:23.307456 15187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:23.323706 15216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:23.323738 15213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:23.324419 15214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:23.325178 15187 server_base.cc:1034] running on GCE node
I20260811 02:57:23.326205 15187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:23.327965 15187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:23.329231 15187 hybrid_clock.cc:648] HybridClock initialized: now 1786417043329148 us; error 64 us; skew 500 ppm
I20260811 02:57:23.329731 15187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:23.332341 15187 webserver.cc:460] Webserver started at http://127.13.94.189:39639/ using document root <none> and password file <none>
I20260811 02:57:23.333122 15187 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:23.333307 15187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:23.340257 15187 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 02:57:23.343962 15223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:23.345777 15187 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:57:23.346055 15187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "d72eb236aa6043eb91a5dfce161d5057"
format_stamp: "Formatted at 2026-08-11 02:57:19 on dist-test-slave-p3t5"
I20260811 02:57:23.347200 15187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:23.385411 15187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:23.386405 15187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:23.386741 15187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:23.412700 15187 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:45939
I20260811 02:57:23.412712 15277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:45939 every 8 connection(s)
I20260811 02:57:23.414508 15187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:57:23.421697 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 15187
I20260811 02:57:23.422446 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:44361
--webserver_interface=127.13.94.188
--webserver_port=36707
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361 with env {}
I20260811 02:57:23.422668 15278 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:23.426331 15278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Bootstrap starting.
I20260811 02:57:23.470765 15278 log.cc:824] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:23.500553 15278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=0 applied=23 ignored=0} inserts{seen=13 ignored=0} mutations{seen=12 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:23.501120 15278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Bootstrap complete.
I20260811 02:57:23.511186 15278 raft_consensus.cc:348] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:23.512460 15278 raft_consensus.cc:729] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72eb236aa6043eb91a5dfce161d5057, State: Initialized, Role: FOLLOWER
I20260811 02:57:23.513283 15278 consensus_queue.cc:260] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 1.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:23.514335 15284 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:23.514681 15284 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:23.515599 15278 sys_catalog.cc:564] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:23.526564 15297 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:23.526703 15297 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: loading cluster ID for follower catalog manager: success
I20260811 02:57:23.530227 15297 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: acquiring CA information for follower catalog manager: success
I20260811 02:57:23.533071 15297 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:23.548489 15187 master_runner.cc:186] Error getting master registration for 127.13.94.188:44361: Network error: Client connection negotiation failed: client connection to 127.13.94.188:44361: connect: Connection refused (error 111),
I20260811 02:57:23.558559 15187 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:57:23.968783 15280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:23.969164 15280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:23.969318 15280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:23.978196 15280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:23.978355 15280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:23.978459 15280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:23.978590 15280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:23.990051 15280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:44361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.94.188
--webserver_port=36707
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:23.991580 15280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:23.993835 15280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:24.001271 15309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:24.001662 15309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:24.003968 15309 leader_election.cc:288] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d72eb236aa6043eb91a5dfce161d5057 (127.13.94.189:45939), e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188:44361)
W20260811 02:57:24.006098 15128 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.94.188:44361: connect: Connection refused (error 111)
W20260811 02:57:24.009608 15308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:24.010630 15306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:24.011202 15305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:24.015813 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 2 candidate_status { last_received { term: 1 index: 23 } } ignore_live_leader: false dest_uuid: "d72eb236aa6043eb91a5dfce161d5057" is_pre_election: true
I20260811 02:57:24.016495 15252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 1.
I20260811 02:57:24.017477 15126 leader_election.cc:302] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92aef555682a43a1a1619b2bc9ba7ce1, d72eb236aa6043eb91a5dfce161d5057; no voters:
W20260811 02:57:24.018028 15128 leader_election.cc:334] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188:44361): Network error: Client connection negotiation failed: client connection to 127.13.94.188:44361: connect: Connection refused (error 111)
I20260811 02:57:24.018183 15309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 02:57:24.018407 15309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:24.018558 15309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:24.018867 15280 server_base.cc:1034] running on GCE node
I20260811 02:57:24.019922 15280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:24.022095 15280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:24.023432 15280 hybrid_clock.cc:648] HybridClock initialized: now 1786417044023332 us; error 56 us; skew 500 ppm
I20260811 02:57:24.024015 15280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:24.027144 15280 webserver.cc:460] Webserver started at http://127.13.94.188:36707/ using document root <none> and password file <none>
I20260811 02:57:24.028052 15280 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:24.028220 15280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:24.035701 15280 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 02:57:24.039292 15319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:24.041007 15280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:24.041219 15280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
uuid: "e3aa822be3af4af4a40b453c83d7d425"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:24.042382 15280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:24.070524 15280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:24.071600 15280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:24.071916 15280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:24.094357 15280 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.188:44361
I20260811 02:57:24.094318 15378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.188:44361 every 8 connection(s)
I20260811 02:57:24.096312 15280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
I20260811 02:57:24.096935 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 15280
I20260811 02:57:24.097958 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:37837
--local_ip_for_outbound_sockets=127.13.94.129
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939
--webserver_port=34685
--webserver_interface=127.13.94.129
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361 with env {}
I20260811 02:57:24.104502 15379 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:24.107650 15309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:24.108660 15309 leader_election.cc:288] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 2 election: Requested vote from peers d72eb236aa6043eb91a5dfce161d5057 (127.13.94.189:45939), e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188:44361)
I20260811 02:57:24.109891 15379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap starting.
I20260811 02:57:24.111351 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 2 candidate_status { last_received { term: 1 index: 23 } } ignore_live_leader: false dest_uuid: "d72eb236aa6043eb91a5dfce161d5057"
I20260811 02:57:24.111670 15252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:24.145772 15379 log.cc:824] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:24.172158 15379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=0 applied=23 ignored=0} inserts{seen=13 ignored=0} mutations{seen=12 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:24.172940 15379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap complete.
I20260811 02:57:24.186439 15379 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:24.187805 15379 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3aa822be3af4af4a40b453c83d7d425, State: Initialized, Role: FOLLOWER
I20260811 02:57:24.188521 15379 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 1.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:24.189563 15385 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:24.190047 15385 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:24.190847 15379 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:24.191161 15353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 2 candidate_status { last_received { term: 1 index: 23 } } ignore_live_leader: false dest_uuid: "e3aa822be3af4af4a40b453c83d7d425"
I20260811 02:57:24.191769 15353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:24.214249 15353 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 2.
I20260811 02:57:24.214356 15252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 2.
I20260811 02:57:24.215348 15128 leader_election.cc:302] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92aef555682a43a1a1619b2bc9ba7ce1, e3aa822be3af4af4a40b453c83d7d425; no voters:
I20260811 02:57:24.215905 15309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:24.220026 15309 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 LEADER]: Becoming Leader. State: Replica: 92aef555682a43a1a1619b2bc9ba7ce1, State: Running, Role: LEADER
I20260811 02:57:24.220703 15309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 1.23, Last appended by leader: 23, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:24.236718 15397 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:24.236848 15397 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: loading cluster ID for follower catalog manager: success
I20260811 02:57:24.238544 15396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:24.239077 15396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:24.240865 15397 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: acquiring CA information for follower catalog manager: success
I20260811 02:57:24.244706 15397 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:24.246227 15404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:24.249846 15404 catalog_manager.cc:669] Loaded metadata for table default.1_1 [id=2b6d1a80a21d49ec8120aeac1c9b2764]
I20260811 02:57:24.250444 15404 catalog_manager.cc:669] Loaded metadata for table default.0_2 [id=5bf294e9599741cab356594a0889edc3]
I20260811 02:57:24.250967 15404 catalog_manager.cc:669] Loaded metadata for table default.0_1 [id=6c64c679309b4f29ab1ba582a3bcff07]
I20260811 02:57:24.251456 15404 catalog_manager.cc:669] Loaded metadata for table default.1_2 [id=acb767fa16b7453da75b503e3c4be4f5]
I20260811 02:57:24.252017 15404 catalog_manager.cc:669] Loaded metadata for table test-workload [id=f0728c3f7ecc49b5ae49a5bd89ce2df8]
I20260811 02:57:24.254366 15404 tablet_loader.cc:96] loaded metadata for tablet 11ebf0753b444c0fb704488c06204383 (table default.0_2 [id=5bf294e9599741cab356594a0889edc3])
I20260811 02:57:24.254860 15404 tablet_loader.cc:96] loaded metadata for tablet a4ff2780d1b54a829f5a65f9b36d10ec (table default.0_1 [id=6c64c679309b4f29ab1ba582a3bcff07])
I20260811 02:57:24.255336 15404 tablet_loader.cc:96] loaded metadata for tablet e55d01af74a84dc391101460f2a9c063 (table default.1_2 [id=acb767fa16b7453da75b503e3c4be4f5])
I20260811 02:57:24.255836 15404 tablet_loader.cc:96] loaded metadata for tablet ee6c637392954d28abe1172c900498b2 (table default.1_1 [id=2b6d1a80a21d49ec8120aeac1c9b2764])
I20260811 02:57:24.256253 15404 tablet_loader.cc:96] loaded metadata for tablet f78e24d030f549fb86183de598e3576b (table test-workload [id=f0728c3f7ecc49b5ae49a5bd89ce2df8])
I20260811 02:57:24.256631 15404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:24.258051 15404 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:24.258196 15404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:24.260234 15404 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:24.261674 15404 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Loaded TSK: 0
I20260811 02:57:24.263007 15404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:24.284127 15143 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53744:
name: "default.0_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:24.284154 15142 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53760:
name: "default.1_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:57:24.556119 15381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:24.556454 15381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:24.556589 15381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:24.568030 15381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:24.568248 15381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:57:24.578034 15381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:37837
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=34685
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:24.579481 15381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:24.581439 15381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:24.596812 15411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:24.597059 15414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:24.598028 15412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:24.598994 15381 server_base.cc:1034] running on GCE node
I20260811 02:57:24.599830 15381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:24.600965 15381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:24.602239 15381 hybrid_clock.cc:648] HybridClock initialized: now 1786417044602152 us; error 62 us; skew 500 ppm
I20260811 02:57:24.602603 15381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:24.605324 15381 webserver.cc:460] Webserver started at http://127.13.94.129:34685/ using document root <none> and password file <none>
I20260811 02:57:24.606168 15381 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:24.606339 15381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:24.611721 15353 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 2 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 1 index: 23. Preceding OpId from leader: term: 2 index: 24. (index mismatch)
I20260811 02:57:24.612702 15381 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:57:24.612802 15396 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260811 02:57:24.616966 15385 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:24.617303 15385 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:24.617388 15422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:24.619343 15381 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:57:24.619606 15381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "42a22db0f41a4489a0dc8ce5315e4428"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:24.621168 15381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:24.621404 15309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:24.621778 15309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:24.622498 15252 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 2 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 1 index: 23. Preceding OpId from leader: term: 2 index: 24. (index mismatch)
I20260811 02:57:24.623148 15385 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:24.623425 15385 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:24.624038 15396 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260811 02:57:24.634517 15431 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:24.634927 15431 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:24.635115 15426 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:24.635659 15426 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:24.645491 15381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:24.646525 15381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:24.646872 15381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:24.648087 15381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:24.650449 15437 ts_tablet_manager.cc:542] Loading tablet metadata (0/5 complete)
I20260811 02:57:24.665839 15381 ts_tablet_manager.cc:579] Loaded tablet metadata (5 total tablets, 5 live tablets)
I20260811 02:57:24.665990 15381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260811 02:57:24.666186 15381 ts_tablet_manager.cc:594] Registering tablets (0/5 complete)
I20260811 02:57:24.670225 15437 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:24.678020 15381 ts_tablet_manager.cc:610] Registered 5 tablets
I20260811 02:57:24.678117 15381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.012s user 0.012s sys 0.000s
I20260811 02:57:24.690531 15437 log.cc:824] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:24.697315 15437 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:24.698487 15437 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:24.700819 15437 ts_tablet_manager.cc:1397] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.031s user 0.016s sys 0.013s
I20260811 02:57:24.720276 15557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:37837 every 8 connection(s)
I20260811 02:57:24.720288 15381 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:37837
I20260811 02:57:24.720329 15437 raft_consensus.cc:348] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.722031 15437 raft_consensus.cc:729] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:24.722416 15381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:24.722963 15437 consensus_queue.cc:260] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.723325 15437 raft_consensus.cc:388] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:24.723557 15437 raft_consensus.cc:482] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:24.723881 15437 raft_consensus.cc:3037] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:24.729805 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 15381
I20260811 02:57:24.734879 15437 raft_consensus.cc:504] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.735697 15437 leader_election.cc:302] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:24.759830 15437 leader_election.cc:288] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:24.759948 15567 raft_consensus.cc:2781] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:24.762849 15560 heartbeater.cc:347] Connected to a master server at 127.13.94.190:38889
I20260811 02:57:24.763247 15560 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:24.763628 15559 heartbeater.cc:347] Connected to a master server at 127.13.94.188:44361
I20260811 02:57:24.763749 15559 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:24.764531 15559 heartbeater.cc:511] Master 127.13.94.188:44361 requested a full tablet report, sending...
I20260811 02:57:24.765430 15560 heartbeater.cc:511] Master 127.13.94.190:38889 requested a full tablet report, sending...
I20260811 02:57:24.767187 15561 heartbeater.cc:347] Connected to a master server at 127.13.94.189:45939
I20260811 02:57:24.767321 15561 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:24.767737 15561 heartbeater.cc:511] Master 127.13.94.189:45939 requested a full tablet report, sending...
I20260811 02:57:24.769881 15567 raft_consensus.cc:686] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:24.770556 15567 consensus_queue.cc:237] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.774001 15437 ts_tablet_manager.cc:1428] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.073s user 0.024s sys 0.023s
I20260811 02:57:24.774526 15437 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:24.782919 15143 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:24.786885 15240 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:24.786710 15143 catalog_manager.cc:5495] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:24.791780 15396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } }
I20260811 02:57:24.792145 15396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:24.796236 15339 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:24.797732 15437 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:24.798341 15437 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:24.799500 15437 ts_tablet_manager.cc:1397] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.006s
I20260811 02:57:24.800719 15437 raft_consensus.cc:348] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.800974 15437 raft_consensus.cc:729] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:24.801297 15437 consensus_queue.cc:260] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.801523 15437 raft_consensus.cc:388] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:24.801661 15437 raft_consensus.cc:482] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:24.801783 15437 raft_consensus.cc:3037] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:24.804034 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:24.805538 15437 raft_consensus.cc:504] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.805908 15437 leader_election.cc:302] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:24.806221 15569 raft_consensus.cc:2781] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:24.806443 15569 raft_consensus.cc:686] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:24.806766 15569 consensus_queue.cc:237] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.810078 15437 leader_election.cc:288] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:24.810658 15437 ts_tablet_manager.cc:1428] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.011s user 0.005s sys 0.003s
I20260811 02:57:24.810953 15437 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:24.812445 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:24.813906 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.187:44947
--webserver_interface=127.13.94.187
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947 with env {}
I20260811 02:57:24.825261 15143 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.94.129:55033
I20260811 02:57:24.831388 15560 heartbeater.cc:503] Master 127.13.94.190:38889 was elected leader, sending a full tablet report...
I20260811 02:57:24.834959 15142 catalog_manager.cc:5495] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:24.840261 15437 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:24.840857 15437 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:24.842173 15437 ts_tablet_manager.cc:1397] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.031s user 0.012s sys 0.019s
I20260811 02:57:24.843099 15437 raft_consensus.cc:348] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.843334 15437 raft_consensus.cc:729] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:24.843604 15437 consensus_queue.cc:260] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.843801 15437 raft_consensus.cc:388] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:24.843921 15437 raft_consensus.cc:482] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:24.844018 15437 raft_consensus.cc:3037] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:24.846316 15437 raft_consensus.cc:504] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.846554 15437 leader_election.cc:302] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:24.846905 15437 leader_election.cc:288] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:24.846963 15569 raft_consensus.cc:2781] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:24.847190 15569 raft_consensus.cc:686] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:24.847402 15437 ts_tablet_manager.cc:1428] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260811 02:57:24.847674 15437 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:24.847510 15569 consensus_queue.cc:237] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.852567 15142 catalog_manager.cc:5495] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:24.866441 15437 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:24.867000 15437 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:24.868033 15437 ts_tablet_manager.cc:1397] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.006s
I20260811 02:57:24.868887 15437 raft_consensus.cc:348] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.869087 15437 raft_consensus.cc:729] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:24.869360 15437 consensus_queue.cc:260] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.869550 15437 raft_consensus.cc:388] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:24.869686 15437 raft_consensus.cc:482] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:24.869808 15437 raft_consensus.cc:3037] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:24.871747 15437 raft_consensus.cc:504] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.872028 15437 leader_election.cc:302] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:24.872292 15437 leader_election.cc:288] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:24.872399 15569 raft_consensus.cc:2781] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:24.872587 15569 raft_consensus.cc:686] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:24.872861 15437 ts_tablet_manager.cc:1428] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:57:24.872869 15569 consensus_queue.cc:237] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.873203 15437 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:24.876362 15142 catalog_manager.cc:5495] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:24.890465 15437 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:24.891038 15437 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:24.892140 15437 ts_tablet_manager.cc:1397] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.004s
I20260811 02:57:24.892947 15437 raft_consensus.cc:348] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.893150 15437 raft_consensus.cc:729] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:24.893447 15437 consensus_queue.cc:260] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.893646 15437 raft_consensus.cc:388] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:24.893785 15437 raft_consensus.cc:482] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:24.893913 15437 raft_consensus.cc:3037] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:24.895846 15437 raft_consensus.cc:504] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.896112 15437 leader_election.cc:302] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:24.896337 15437 leader_election.cc:288] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:24.896378 15569 raft_consensus.cc:2781] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:24.896528 15569 raft_consensus.cc:686] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:24.896765 15569 consensus_queue.cc:237] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:24.896867 15437 ts_tablet_manager.cc:1428] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260811 02:57:24.900063 15142 catalog_manager.cc:5495] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
W20260811 02:57:25.254768 15579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:25.255113 15579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:25.255192 15579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:25.263484 15579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:25.263609 15579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:25.263652 15579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:25.263702 15579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:25.273546 15579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.187:44947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.13.94.187
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:25.275045 15579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:25.276878 15579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:25.292896 15590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:25.293674 15591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:25.295316 15593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:25.302294 15579 server_base.cc:1034] running on GCE node
I20260811 02:57:25.303869 15579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:25.326011 15579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:25.333665 15579 hybrid_clock.cc:648] HybridClock initialized: now 1786417045333437 us; error 133 us; skew 500 ppm
I20260811 02:57:25.334395 15579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:25.352267 15142 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53798:
name: "default.1_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:25.352267 15143 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53794:
name: "default.0_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:25.355983 15579 webserver.cc:460] Webserver started at http://127.13.94.187:35587/ using document root <none> and password file <none>
I20260811 02:57:25.358409 15579 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:25.358618 15579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:25.359133 15579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:25.363549 15579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/instance:
uuid: "49ac0879298948648d31096ff564f34e"
format_stamp: "Formatted at 2026-08-11 02:57:25 on dist-test-slave-p3t5"
I20260811 02:57:25.364713 15579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal/instance:
uuid: "49ac0879298948648d31096ff564f34e"
format_stamp: "Formatted at 2026-08-11 02:57:25 on dist-test-slave-p3t5"
I20260811 02:57:25.375694 15579 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.011s sys 0.001s
I20260811 02:57:25.388543 15625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:25.390833 15579 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.007s sys 0.003s
I20260811 02:57:25.391084 15579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
uuid: "49ac0879298948648d31096ff564f34e"
format_stamp: "Formatted at 2026-08-11 02:57:25 on dist-test-slave-p3t5"
I20260811 02:57:25.391395 15579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:25.396698 15484 tablet_service.cc:1467] Processing CreateTablet for tablet 420ed2b9216647498d9a2c956475ce4c (DEFAULT_TABLE table=default.0_4 [id=1dd6d5be9a9f4a6ba48fee38b5e87d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:25.397746 15484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420ed2b9216647498d9a2c956475ce4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:25.396667 15485 tablet_service.cc:1467] Processing CreateTablet for tablet f2e8bb100c3f41d5b74490a795fe638e (DEFAULT_TABLE table=default.1_4 [id=1765740dfbf5417e90b87056985f59e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:25.398272 15485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2e8bb100c3f41d5b74490a795fe638e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:25.407974 15633 tablet_bootstrap.cc:492] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:25.410166 15633 tablet_bootstrap.cc:654] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:25.412371 15633 tablet_bootstrap.cc:492] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:25.412515 15633 ts_tablet_manager.cc:1397] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:57:25.413237 15633 raft_consensus.cc:348] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:25.413447 15633 raft_consensus.cc:374] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:25.413542 15633 raft_consensus.cc:729] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:25.413805 15633 consensus_queue.cc:260] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:25.414014 15633 raft_consensus.cc:388] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:25.414083 15633 raft_consensus.cc:482] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:25.414177 15633 raft_consensus.cc:3037] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:25.416882 15633 raft_consensus.cc:504] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:25.417148 15633 leader_election.cc:302] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:25.421921 15633 leader_election.cc:288] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:25.422132 15636 raft_consensus.cc:2781] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:25.429567 15579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:25.430194 15633 ts_tablet_manager.cc:1428] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.018s user 0.006s sys 0.010s
I20260811 02:57:25.430532 15636 raft_consensus.cc:686] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:25.430814 15579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:25.431020 15636 consensus_queue.cc:237] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:25.431305 15633 tablet_bootstrap.cc:492] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:25.431694 15579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:25.433570 15633 tablet_bootstrap.cc:654] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:25.438715 15143 catalog_manager.cc:5495] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:25.448935 15633 tablet_bootstrap.cc:492] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:25.449211 15633 ts_tablet_manager.cc:1397] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.002s
I20260811 02:57:25.450356 15633 raft_consensus.cc:348] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:25.450676 15633 raft_consensus.cc:374] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:25.450773 15633 raft_consensus.cc:729] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:25.451076 15633 consensus_queue.cc:260] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:25.451316 15633 raft_consensus.cc:388] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:25.451442 15633 raft_consensus.cc:482] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:25.451593 15633 raft_consensus.cc:3037] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:25.455443 15633 raft_consensus.cc:504] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:25.455742 15633 leader_election.cc:302] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:25.455964 15633 leader_election.cc:288] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:25.456442 15633 ts_tablet_manager.cc:1428] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 02:57:25.456799 15637 raft_consensus.cc:2781] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:25.457000 15637 raft_consensus.cc:686] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:25.457281 15637 consensus_queue.cc:237] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:25.460284 15143 catalog_manager.cc:5495] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:25.489923 15579 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.187:44947
I20260811 02:57:25.489923 15693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.187:44947 every 8 connection(s)
I20260811 02:57:25.491474 15579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/info.pb
I20260811 02:57:25.495558 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 15579
I20260811 02:57:25.496129 15694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:25.505316 15694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:25.523226 15694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:25.531466 15694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:25.538537 15694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:25.553125 15694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Bootstrap starting.
I20260811 02:57:25.556622 15694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:25.557893 15694 log.cc:824] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:25.561698 15694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: No bootstrap required, opened a new log
I20260811 02:57:25.570458 15694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } }
I20260811 02:57:25.570920 15694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:25.571051 15694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49ac0879298948648d31096ff564f34e, State: Initialized, Role: FOLLOWER
I20260811 02:57:25.571671 15694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } }
I20260811 02:57:25.572686 15700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } } }
I20260811 02:57:25.573060 15700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:25.573800 15694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:25.583392 15712 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:25.583559 15712 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:25.591203 15579 master_runner.cc:428] Detected that this master 49ac0879298948648d31096ff564f34e is joining an existing cluster
I20260811 02:57:25.591326 15579 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.94.187:44947
I20260811 02:57:25.617877 15700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:25.618290 15700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } }
I20260811 02:57:25.642517 15700 leader_election.cc:288] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92aef555682a43a1a1619b2bc9ba7ce1 (127.13.94.190:38889), d72eb236aa6043eb91a5dfce161d5057 (127.13.94.189:45939), e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188:44361)
I20260811 02:57:25.643317 15353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49ac0879298948648d31096ff564f34e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3aa822be3af4af4a40b453c83d7d425" is_pre_election: true
I20260811 02:57:25.644686 15156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49ac0879298948648d31096ff564f34e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" is_pre_election: true
I20260811 02:57:25.646021 15630 leader_election.cc:302] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49ac0879298948648d31096ff564f34e; no voters: 92aef555682a43a1a1619b2bc9ba7ce1, e3aa822be3af4af4a40b453c83d7d425
I20260811 02:57:25.646800 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49ac0879298948648d31096ff564f34e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72eb236aa6043eb91a5dfce161d5057" is_pre_election: true
I20260811 02:57:25.647770 15700 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 02:57:25.657440 15143 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.94.187:44947: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "49ac0879298948648d31096ff564f34e"
member_type: NON_VOTER
last_known_addr {
host: "127.13.94.187"
port: 44947
}
attrs {
promote: true
}
}
dest_uuid: "92aef555682a43a1a1619b2bc9ba7ce1"
cas_config_opid_index: 18
I20260811 02:57:25.659076 15143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 2.35, Last appended by leader: 23, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } }
I20260811 02:57:25.661968 15353 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 2 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20260811 02:57:25.661970 15252 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 2 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20260811 02:57:25.662350 15668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 0 FOLLOWER]: Advancing to term 2
I20260811 02:57:25.662881 15575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20260811 02:57:25.663506 15575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20260811 02:57:25.666482 15668 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20260811 02:57:25.666936 15309 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 LEADER]: Committing config change with OpId 2.36: config changed from index 18 to 36, NON_VOTER 49ac0879298948648d31096ff564f34e (127.13.94.187) added. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:25.668216 15575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:25.667689 15353 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 18 to 36, NON_VOTER 49ac0879298948648d31096ff564f34e (127.13.94.187) added. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:25.672746 15252 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 18 to 36, NON_VOTER 49ac0879298948648d31096ff564f34e (127.13.94.187) added. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:25.675554 15309 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.94.187:44947
I20260811 02:57:25.676339 15579 master.cc:537] Master@127.13.94.187:44947 shutting down...
I20260811 02:57:25.679991 15396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:25.680330 15396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:25.684839 15575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:25.685348 15575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:25.685938 15575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:25.686497 15575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:25.687415 15725 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:25.687978 15725 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:25.693553 15722 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:25.694067 15722 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 02:57:25.695273 15143 master.cc:648] Remote error: unable to get registration information for peer 49ac0879298948648d31096ff564f34e: Service unavailable: service kudu.master.MasterService not registered on Master
I20260811 02:57:25.696177 15668 raft_consensus.cc:858] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } }.
Pending config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } }:
New config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
W20260811 02:57:25.696408 15668 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 18' and following 18 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } }.
Pending config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } }:
New config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:25.703271 15668 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 NON_PARTICIPANT]: Committing config change with OpId 1.15: config changed from index -1 to 15, NON_VOTER e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:25.708438 15726 mvcc.cc:205] Tried to move back new op lower bound from 7317164198061416448 to 7317164194469126144. Current Snapshot: MvccSnapshot[applied={T|T < 7317164198061416448 or (T in {7317164198061416448})}]
I20260811 02:57:25.709745 15700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:25.710045 15700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:57:25.709899 15728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:25.710402 15728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:57:25.710865 15732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:25.711141 15732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:57:25.713034 15579 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 02:57:25.713686 15579 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 02:57:25.713864 15579 tablet_replica.cc:330] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: stopping tablet replica
I20260811 02:57:25.714293 15309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:25.714649 15309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
W20260811 02:57:25.716774 15129 consensus_peers.cc:487] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 -> Peer 49ac0879298948648d31096ff564f34e (127.13.94.187:44947): Couldn't send request to peer 49ac0879298948648d31096ff564f34e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 02:57:25.732782 15579 master.cc:552] Master@127.13.94.187:44947 shutdown complete.
I20260811 02:57:25.732901 15579 master_runner.cc:305] Clearing existing system tablet
I20260811 02:57:25.733353 15579 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:25.733469 15579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:25.736780 15579 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:57:25.738709 15735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:25.739228 15579 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:25.739486 15579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:25.739591 15579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
uuid: "49ac0879298948648d31096ff564f34e"
format_stamp: "Formatted at 2026-08-11 02:57:25 on dist-test-slave-p3t5"
I20260811 02:57:25.744347 15579 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:25.752525 15579 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 02:57:25.752846 15579 log.cc:1196] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Deleting WAL directory at /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260811 02:57:25.753546 15579 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Deleting consensus metadata
I20260811 02:57:25.753906 15579 master_runner.cc:315] Copying system tablet from 127.13.94.190:38889
I20260811 02:57:25.764981 15579 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: tablet copy: Beginning tablet copy session from remote peer at address 127.13.94.190:38889
I20260811 02:57:25.773211 15168 tablet_copy_service.cc:140] P 92aef555682a43a1a1619b2bc9ba7ce1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 49ac0879298948648d31096ff564f34e ({username='slave'} at 127.0.0.1:50534)
I20260811 02:57:25.773511 15168 tablet_copy_service.cc:161] P 92aef555682a43a1a1619b2bc9ba7ce1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 49ac0879298948648d31096ff564f34e at {username='slave'} at 127.0.0.1:50534: session id = 49ac0879298948648d31096ff564f34e-00000000000000000000000000000000
I20260811 02:57:25.777261 15168 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 02:57:25.779614 15579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 02:57:25.784447 15579 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: tablet copy: Starting download of 0 data blocks...
I20260811 02:57:25.784789 15579 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: tablet copy: Starting download of 1 WAL segments...
I20260811 02:57:25.787680 15579 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 02:57:25.791710 15168 tablet_copy_service.cc:342] P 92aef555682a43a1a1619b2bc9ba7ce1: Request end of tablet copy session 49ac0879298948648d31096ff564f34e-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:50534
I20260811 02:57:25.791987 15168 tablet_copy_service.cc:433] P 92aef555682a43a1a1619b2bc9ba7ce1: ending tablet copy session 49ac0879298948648d31096ff564f34e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 49ac0879298948648d31096ff564f34e
I20260811 02:57:25.803875 15579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:25.807257 15743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:25.807981 15742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:25.809865 15745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:25.814013 15579 server_base.cc:1034] running on GCE node
I20260811 02:57:25.814432 15579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:25.815229 15579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:25.816439 15579 hybrid_clock.cc:648] HybridClock initialized: now 1786417045816385 us; error 57 us; skew 500 ppm
I20260811 02:57:25.816764 15579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:25.818625 15579 webserver.cc:460] Webserver started at http://127.13.94.187:41091/ using document root <none> and password file <none>
I20260811 02:57:25.819000 15579 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:25.819115 15579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:25.822043 15579 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 02:57:25.823892 15751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:25.824359 15579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 02:57:25.824533 15579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
uuid: "49ac0879298948648d31096ff564f34e"
format_stamp: "Formatted at 2026-08-11 02:57:25 on dist-test-slave-p3t5"
I20260811 02:57:25.824697 15579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260811 02:57:25.832191 15143 master.cc:648] Network error: unable to get registration information for peer 49ac0879298948648d31096ff564f34e: Client connection negotiation failed: client connection to 127.13.94.187:44947: connect: Connection refused (error 111)
I20260811 02:57:25.837811 15579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:25.838577 15579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:25.859088 15579 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.187:44947
I20260811 02:57:25.860262 15579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/info.pb
I20260811 02:57:25.859112 15807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.187:44947 every 8 connection(s)
I20260811 02:57:25.864801 15808 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:25.867408 15808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Bootstrap starting.
I20260811 02:57:25.904670 15808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Bootstrap replayed 1/1 log segments. Stats: ops{read=36 overwritten=0 applied=36 ignored=0} inserts{seen=17 ignored=0} mutations{seen=21 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:25.905261 15808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Bootstrap complete.
I20260811 02:57:25.906458 15808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } }
I20260811 02:57:25.906805 15808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 49ac0879298948648d31096ff564f34e, State: Initialized, Role: LEARNER
I20260811 02:57:25.907109 15808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 36, Last appended: 2.36, Last appended by leader: 36, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } }
I20260811 02:57:25.908114 15811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:25.908531 15808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:25.908537 15811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:25.918720 15822 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:25.918861 15822 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: loading cluster ID for follower catalog manager: success
I20260811 02:57:25.923125 15822 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: acquiring CA information for follower catalog manager: success
I20260811 02:57:25.926311 15822 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:26.198535 15396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:26.198959 15396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:26.199910 15781 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 LEARNER]: Deduplicated request from leader. Original: 1.17->[1.18-2.36] Dedup: 2.36->[]
I20260811 02:57:26.200371 15811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: NON_VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: true } } }
I20260811 02:57:26.200687 15811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:26.201102 15396 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: attempting to promote NON_VOTER 49ac0879298948648d31096ff564f34e to VOTER
I20260811 02:57:26.202097 15396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 2.36, Last appended by leader: 23, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:26.204294 15252 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 2 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 2 index: 36. Preceding OpId from leader: term: 2 index: 37. (index mismatch)
I20260811 02:57:26.204294 15353 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 2 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 2 index: 36. Preceding OpId from leader: term: 2 index: 37. (index mismatch)
I20260811 02:57:26.204594 15781 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 LEARNER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 2 index: 36. Preceding OpId from leader: term: 2 index: 37. (index mismatch)
I20260811 02:57:26.204924 15575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20260811 02:57:26.205197 15396 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20260811 02:57:26.205572 15824 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20260811 02:57:26.208334 15824 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 LEADER]: Committing config change with OpId 2.37: config changed from index 36 to 37, 49ac0879298948648d31096ff564f34e (127.13.94.187) changed from NON_VOTER to VOTER. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.209201 15252 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 2 FOLLOWER]: Committing config change with OpId 2.37: config changed from index 36 to 37, 49ac0879298948648d31096ff564f34e (127.13.94.187) changed from NON_VOTER to VOTER. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.211585 15824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.211848 15824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:26.211820 15309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.212193 15309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:26.212160 15824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.212495 15824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:26.213177 15353 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 2 FOLLOWER]: Committing config change with OpId 2.37: config changed from index 36 to 37, 49ac0879298948648d31096ff564f34e (127.13.94.187) changed from NON_VOTER to VOTER. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.214771 15826 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.215090 15826 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:26.215412 15309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.215725 15309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:26.216078 15781 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 FOLLOWER]: Committing config change with OpId 2.37: config changed from index 36 to 37, 49ac0879298948648d31096ff564f34e (127.13.94.187) changed from NON_VOTER to VOTER. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.217299 15827 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.217667 15827 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:26.219561 15811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:26.219832 15811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:26.492630 15143 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50548:
name: "default.1_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:26.522871 15485 tablet_service.cc:1467] Processing CreateTablet for tablet 0c91c12fe10e45f8a8dacdadf9a332dc (DEFAULT_TABLE table=default.1_5 [id=aea54d2174504fa89b4df2ff08a53688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:26.523389 15485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c91c12fe10e45f8a8dacdadf9a332dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:26.532653 15856 tablet_bootstrap.cc:492] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:26.535519 15856 tablet_bootstrap.cc:654] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:26.535647 15143 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50554:
name: "default.0_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:26.540169 15856 tablet_bootstrap.cc:492] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:26.540345 15856 ts_tablet_manager.cc:1397] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 02:57:26.542215 15856 raft_consensus.cc:348] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:26.543509 15856 raft_consensus.cc:374] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:26.543700 15856 raft_consensus.cc:729] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:26.544054 15856 consensus_queue.cc:260] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:26.544363 15856 raft_consensus.cc:388] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:26.544514 15856 raft_consensus.cc:482] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:26.544696 15856 raft_consensus.cc:3037] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:26.548296 15856 raft_consensus.cc:504] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:26.548717 15856 leader_election.cc:302] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:26.550287 15856 leader_election.cc:288] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:26.552507 15485 tablet_service.cc:1467] Processing CreateTablet for tablet 3a186dc26d0246568bcd1be33c1aa781 (DEFAULT_TABLE table=default.0_5 [id=a539b8205c734c08b5cf1e5e1250b1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:26.553916 15485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a186dc26d0246568bcd1be33c1aa781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:26.554612 15858 raft_consensus.cc:2781] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:26.559139 15858 raft_consensus.cc:686] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:26.560281 15858 consensus_queue.cc:237] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:26.567535 15856 ts_tablet_manager.cc:1428] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.027s user 0.015s sys 0.009s
I20260811 02:57:26.567968 15856 tablet_bootstrap.cc:492] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:26.569942 15856 tablet_bootstrap.cc:654] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:26.571369 15143 catalog_manager.cc:5495] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:26.582571 15856 tablet_bootstrap.cc:492] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:26.582789 15856 ts_tablet_manager.cc:1397] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.005s
I20260811 02:57:26.583658 15856 raft_consensus.cc:348] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:26.583900 15856 raft_consensus.cc:374] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:26.583985 15856 raft_consensus.cc:729] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:26.584234 15856 consensus_queue.cc:260] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:26.584430 15856 raft_consensus.cc:388] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:26.584549 15856 raft_consensus.cc:482] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:26.584641 15856 raft_consensus.cc:3037] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:26.587389 15856 raft_consensus.cc:504] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:26.587658 15856 leader_election.cc:302] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:26.587905 15858 raft_consensus.cc:2781] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:26.588092 15858 raft_consensus.cc:686] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:26.588315 15858 consensus_queue.cc:237] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:26.595079 15856 leader_election.cc:288] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:26.595726 15856 ts_tablet_manager.cc:1428] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20260811 02:57:26.598865 15141 catalog_manager.cc:5495] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:26.694707 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 15381
I20260811 02:57:26.702600 15553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:26.816273 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 15381
I20260811 02:57:26.839761 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 15094
I20260811 02:57:26.840961 15179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:26.961006 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 15094
I20260811 02:57:26.988003 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 15187
I20260811 02:57:26.989403 15273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:27.109155 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 15187
I20260811 02:57:27.135618 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 15280
I20260811 02:57:27.136797 15373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:27.246490 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 15280
I20260811 02:57:27.272414 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 15579
I20260811 02:57:27.273556 15803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:27.388182 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 15579
I20260811 02:57:27.415220 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:38889
--webserver_interface=127.13.94.190
--webserver_port=43213
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947 with env {}
W20260811 02:57:27.794727 15877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:27.795116 15877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:27.795256 15877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:27.803443 15877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:27.803591 15877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:27.803665 15877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:27.803755 15877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:27.814471 15877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=43213
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:27.815970 15877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:27.818459 15877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:27.830747 15900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:27.831521 15901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:27.832930 15903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:27.833602 15877 server_base.cc:1034] running on GCE node
I20260811 02:57:27.834613 15877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:27.836109 15877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:27.837357 15877 hybrid_clock.cc:648] HybridClock initialized: now 1786417047837273 us; error 63 us; skew 500 ppm
I20260811 02:57:27.837823 15877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:27.840534 15877 webserver.cc:460] Webserver started at http://127.13.94.190:43213/ using document root <none> and password file <none>
I20260811 02:57:27.841293 15877 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:27.841403 15877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:27.847163 15877 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:27.851119 15909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:27.852741 15877 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:27.852942 15877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "92aef555682a43a1a1619b2bc9ba7ce1"
format_stamp: "Formatted at 2026-08-11 02:57:18 on dist-test-slave-p3t5"
I20260811 02:57:27.854046 15877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:27.872308 15877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:27.873415 15877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:27.873826 15877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:27.894865 15877 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:38889
I20260811 02:57:27.894975 15964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:38889 every 8 connection(s)
I20260811 02:57:27.896987 15877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:27.898217 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 15877
I20260811 02:57:27.899080 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:45939
--webserver_interface=127.13.94.189
--webserver_port=39639
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947 with env {}
I20260811 02:57:27.914860 15965 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:27.921169 15965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Bootstrap starting.
I20260811 02:57:27.947938 15965 log.cc:824] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:27.992900 15965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Bootstrap replayed 1/1 log segments. Stats: ops{read=45 overwritten=0 applied=45 ignored=0} inserts{seen=21 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:27.993558 15965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Bootstrap complete.
I20260811 02:57:28.002609 15965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:28.003620 15965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92aef555682a43a1a1619b2bc9ba7ce1, State: Initialized, Role: FOLLOWER
I20260811 02:57:28.004263 15965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 45, Last appended: 2.45, Last appended by leader: 45, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:28.005311 15971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:28.005883 15971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:28.006493 15965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:28.026067 15983 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:28.026211 15983 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: loading cluster ID for follower catalog manager: success
I20260811 02:57:28.030784 15983 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: acquiring CA information for follower catalog manager: success
I20260811 02:57:28.034686 15983 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:28.038668 15877 master_runner.cc:186] Error getting master registration for 127.13.94.189:45939: Network error: Client connection negotiation failed: client connection to 127.13.94.189:45939: connect: Connection refused (error 111),
I20260811 02:57:28.046881 15877 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:57:28.312247 15968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:28.312656 15968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:28.312800 15968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:28.320761 15968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:28.320933 15968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:28.321044 15968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:28.321149 15968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:28.331056 15968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:45939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--webserver_port=39639
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:28.332610 15968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:28.334823 15968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:28.350570 15992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:28.352378 15991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:28.352847 15994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:28.353299 15968 server_base.cc:1034] running on GCE node
I20260811 02:57:28.354439 15968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:28.356170 15968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:28.357455 15968 hybrid_clock.cc:648] HybridClock initialized: now 1786417048357367 us; error 71 us; skew 500 ppm
I20260811 02:57:28.357936 15968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:28.360900 15968 webserver.cc:460] Webserver started at http://127.13.94.189:39639/ using document root <none> and password file <none>
I20260811 02:57:28.361845 15968 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:28.362020 15968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:28.368480 15968 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:57:28.372256 16000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:28.374136 15968 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:57:28.374404 15968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "d72eb236aa6043eb91a5dfce161d5057"
format_stamp: "Formatted at 2026-08-11 02:57:19 on dist-test-slave-p3t5"
I20260811 02:57:28.375530 15968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:28.397727 15968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:28.398777 15968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:28.399142 15968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:28.423771 15968 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:45939
I20260811 02:57:28.423799 16058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:45939 every 8 connection(s)
I20260811 02:57:28.425554 15968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:57:28.426998 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 15968
I20260811 02:57:28.427670 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:44361
--webserver_interface=127.13.94.188
--webserver_port=36707
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947 with env {}
I20260811 02:57:28.440642 16059 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:28.447151 16059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Bootstrap starting.
I20260811 02:57:28.477591 16059 log.cc:824] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:28.527140 16059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Bootstrap replayed 1/1 log segments. Stats: ops{read=45 overwritten=0 applied=45 ignored=0} inserts{seen=21 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:28.527872 16059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Bootstrap complete.
I20260811 02:57:28.540506 16059 raft_consensus.cc:348] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:28.541980 16059 raft_consensus.cc:729] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72eb236aa6043eb91a5dfce161d5057, State: Initialized, Role: FOLLOWER
I20260811 02:57:28.542755 16059 consensus_queue.cc:260] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 45, Last appended: 2.45, Last appended by leader: 45, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:28.543813 16066 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:28.544250 16066 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:28.545197 16059 sys_catalog.cc:564] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:28.574185 16078 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:28.574331 16078 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: loading cluster ID for follower catalog manager: success
I20260811 02:57:28.582803 16078 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: acquiring CA information for follower catalog manager: success
I20260811 02:57:28.589761 16078 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:28.603223 15968 master_runner.cc:186] Error getting master registration for 127.13.94.188:44361: Network error: Client connection negotiation failed: client connection to 127.13.94.188:44361: connect: Connection refused (error 111),
I20260811 02:57:28.610083 15968 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:57:28.863489 16061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:28.863830 16061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:28.863981 16061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:28.873083 16061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:28.873205 16061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:28.873281 16061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:28.873384 16061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:28.883610 16061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:44361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.94.188
--webserver_port=36707
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:28.885103 16061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:28.887176 16061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:28.902997 16089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:28.904223 16088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:28.907773 16061 server_base.cc:1034] running on GCE node
W20260811 02:57:28.909919 16091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:28.911270 16061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:28.913069 16061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:28.914944 16061 hybrid_clock.cc:648] HybridClock initialized: now 1786417048914836 us; error 63 us; skew 500 ppm
I20260811 02:57:28.915424 16061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:28.919152 16061 webserver.cc:460] Webserver started at http://127.13.94.188:36707/ using document root <none> and password file <none>
I20260811 02:57:28.919971 16061 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:28.920152 16061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:28.926395 16061 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:57:28.929847 16097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:28.931423 16061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:57:28.931630 16061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
uuid: "e3aa822be3af4af4a40b453c83d7d425"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:28.932667 16061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:28.951463 16061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:28.952423 16061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:28.952740 16061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:28.977608 16061 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.188:44361
I20260811 02:57:28.977651 16151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.188:44361 every 8 connection(s)
I20260811 02:57:28.979269 16061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
I20260811 02:57:28.984437 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 16061
I20260811 02:57:28.985069 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.187:44947
--webserver_interface=127.13.94.187
--webserver_port=35587
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947 with env {}
I20260811 02:57:28.986930 16152 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:28.990670 16152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap starting.
I20260811 02:57:29.013108 16152 log.cc:824] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:29.065855 16152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap replayed 1/1 log segments. Stats: ops{read=45 overwritten=0 applied=45 ignored=0} inserts{seen=21 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:29.066514 16152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap complete.
I20260811 02:57:29.076014 16152 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:29.077072 16152 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3aa822be3af4af4a40b453c83d7d425, State: Initialized, Role: FOLLOWER
I20260811 02:57:29.077893 16152 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 45, Last appended: 2.45, Last appended by leader: 45, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:29.079113 16158 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:29.079624 16158 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:29.080117 16152 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:29.101634 16171 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:29.101764 16171 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: loading cluster ID for follower catalog manager: success
I20260811 02:57:29.106230 16171 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: acquiring CA information for follower catalog manager: success
I20260811 02:57:29.109694 16171 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:29.125079 16061 master_runner.cc:186] Error getting master registration for 127.13.94.187:44947: Network error: Client connection negotiation failed: client connection to 127.13.94.187:44947: connect: Connection refused (error 111),
I20260811 02:57:29.132483 16061 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:57:29.387001 16154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:29.387539 16154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:29.387729 16154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:29.396850 16154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:29.397037 16154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:29.397147 16154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:29.397251 16154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:29.408046 16154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.187:44947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.13.94.187
--webserver_port=35587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:29.409729 16154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:29.412106 16154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:29.427156 16182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:29.427354 16180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:29.427783 16179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:29.430342 16154 server_base.cc:1034] running on GCE node
I20260811 02:57:29.431628 16154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:29.433645 16154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:29.434967 16154 hybrid_clock.cc:648] HybridClock initialized: now 1786417049434879 us; error 56 us; skew 500 ppm
I20260811 02:57:29.435402 16154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:29.438916 16154 webserver.cc:460] Webserver started at http://127.13.94.187:35587/ using document root <none> and password file <none>
I20260811 02:57:29.439934 16154 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:29.440106 16154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:29.450649 16154 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.005s sys 0.000s
I20260811 02:57:29.454949 16189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:29.456892 16154 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 02:57:29.457125 16154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
uuid: "49ac0879298948648d31096ff564f34e"
format_stamp: "Formatted at 2026-08-11 02:57:25 on dist-test-slave-p3t5"
I20260811 02:57:29.458705 16154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:29.485607 16154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:29.486645 16154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:29.486989 16154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:29.521667 16154 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.187:44947
I20260811 02:57:29.521684 16245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.187:44947 every 8 connection(s)
I20260811 02:57:29.523428 16154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/info.pb
I20260811 02:57:29.530722 16248 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:29.532434 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 16154
I20260811 02:57:29.533099 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:37837
--local_ip_for_outbound_sockets=127.13.94.129
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939
--webserver_port=34685
--webserver_interface=127.13.94.129
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947 with env {}
I20260811 02:57:29.534582 16248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Bootstrap starting.
I20260811 02:57:29.567777 16248 log.cc:824] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:29.618324 16248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Bootstrap replayed 1/1 log segments. Stats: ops{read=45 overwritten=0 applied=45 ignored=0} inserts{seen=21 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:29.619134 16248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Bootstrap complete.
I20260811 02:57:29.632968 16248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:29.634380 16248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49ac0879298948648d31096ff564f34e, State: Initialized, Role: FOLLOWER
I20260811 02:57:29.635274 16248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 45, Last appended: 2.45, Last appended by leader: 45, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:29.636281 16252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:29.636871 16252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:29.637639 16248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:29.661875 16263 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:29.665122 16263 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: loading cluster ID for follower catalog manager: success
I20260811 02:57:29.668694 16263 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: acquiring CA information for follower catalog manager: success
I20260811 02:57:29.672490 16263 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:29.734095 16154 master_runner.cc:263] No leader master found from master 49ac0879298948648d31096ff564f34e
I20260811 02:57:29.734282 16154 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:57:29.803771 16271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:29.804139 16271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:29.806874 16271 leader_election.cc:288] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72eb236aa6043eb91a5dfce161d5057 (127.13.94.189:45939), e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188:44361), 49ac0879298948648d31096ff564f34e (127.13.94.187:44947)
I20260811 02:57:29.831923 16124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 3 candidate_status { last_received { term: 2 index: 45 } } ignore_live_leader: false dest_uuid: "e3aa822be3af4af4a40b453c83d7d425" is_pre_election: true
I20260811 02:57:29.832600 16124 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 2.
I20260811 02:57:29.835366 16029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 3 candidate_status { last_received { term: 2 index: 45 } } ignore_live_leader: false dest_uuid: "d72eb236aa6043eb91a5dfce161d5057" is_pre_election: true
I20260811 02:57:29.836032 16029 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 2.
I20260811 02:57:29.836699 15910 leader_election.cc:302] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92aef555682a43a1a1619b2bc9ba7ce1, d72eb236aa6043eb91a5dfce161d5057, e3aa822be3af4af4a40b453c83d7d425; no voters:
I20260811 02:57:29.837296 16271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 02:57:29.837502 16271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:29.837637 16271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:29.848111 16217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 3 candidate_status { last_received { term: 2 index: 45 } } ignore_live_leader: false dest_uuid: "49ac0879298948648d31096ff564f34e" is_pre_election: true
I20260811 02:57:29.848969 16217 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 2.
I20260811 02:57:29.852494 16271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:29.855896 16124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 3 candidate_status { last_received { term: 2 index: 45 } } ignore_live_leader: false dest_uuid: "e3aa822be3af4af4a40b453c83d7d425"
I20260811 02:57:29.855834 16029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 3 candidate_status { last_received { term: 2 index: 45 } } ignore_live_leader: false dest_uuid: "d72eb236aa6043eb91a5dfce161d5057"
I20260811 02:57:29.856166 16124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:29.856205 16029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:29.862249 16124 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 3.
I20260811 02:57:29.862807 16029 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 3.
I20260811 02:57:29.863153 16271 leader_election.cc:288] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 3 election: Requested vote from peers d72eb236aa6043eb91a5dfce161d5057 (127.13.94.189:45939), e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188:44361), 49ac0879298948648d31096ff564f34e (127.13.94.187:44947)
I20260811 02:57:29.863862 16217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 3 candidate_status { last_received { term: 2 index: 45 } } ignore_live_leader: false dest_uuid: "49ac0879298948648d31096ff564f34e"
I20260811 02:57:29.864194 16217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:29.864436 15910 leader_election.cc:302] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92aef555682a43a1a1619b2bc9ba7ce1, d72eb236aa6043eb91a5dfce161d5057, e3aa822be3af4af4a40b453c83d7d425; no voters:
I20260811 02:57:29.864904 16271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:29.871395 16217 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 3.
I20260811 02:57:29.872543 16271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 LEADER]: Becoming Leader. State: Replica: 92aef555682a43a1a1619b2bc9ba7ce1, State: Running, Role: LEADER
I20260811 02:57:29.873286 16271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 45, Committed index: 45, Last appended: 2.45, Last appended by leader: 45, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } }
I20260811 02:57:29.876442 16275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:29.876927 16275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:29.878710 16277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:29.882169 16277 catalog_manager.cc:669] Loaded metadata for table default.1_4 [id=1765740dfbf5417e90b87056985f59e1]
I20260811 02:57:29.882718 16277 catalog_manager.cc:669] Loaded metadata for table default.0_4 [id=1dd6d5be9a9f4a6ba48fee38b5e87d63]
I20260811 02:57:29.883107 16277 catalog_manager.cc:669] Loaded metadata for table default.1_1 [id=2b6d1a80a21d49ec8120aeac1c9b2764]
I20260811 02:57:29.883492 16277 catalog_manager.cc:669] Loaded metadata for table default.0_2 [id=5bf294e9599741cab356594a0889edc3]
I20260811 02:57:29.883862 16277 catalog_manager.cc:669] Loaded metadata for table default.0_1 [id=6c64c679309b4f29ab1ba582a3bcff07]
I20260811 02:57:29.884253 16277 catalog_manager.cc:669] Loaded metadata for table default.0_5 [id=a539b8205c734c08b5cf1e5e1250b1b6]
I20260811 02:57:29.884619 16277 catalog_manager.cc:669] Loaded metadata for table default.1_2 [id=acb767fa16b7453da75b503e3c4be4f5]
I20260811 02:57:29.885022 16277 catalog_manager.cc:669] Loaded metadata for table default.1_5 [id=aea54d2174504fa89b4df2ff08a53688]
I20260811 02:57:29.885402 16277 catalog_manager.cc:669] Loaded metadata for table test-workload [id=f0728c3f7ecc49b5ae49a5bd89ce2df8]
I20260811 02:57:29.887956 16277 tablet_loader.cc:96] loaded metadata for tablet 0c91c12fe10e45f8a8dacdadf9a332dc (table default.1_5 [id=aea54d2174504fa89b4df2ff08a53688])
I20260811 02:57:29.888389 16277 tablet_loader.cc:96] loaded metadata for tablet 11ebf0753b444c0fb704488c06204383 (table default.0_2 [id=5bf294e9599741cab356594a0889edc3])
I20260811 02:57:29.888748 16277 tablet_loader.cc:96] loaded metadata for tablet 3a186dc26d0246568bcd1be33c1aa781 (table default.0_5 [id=a539b8205c734c08b5cf1e5e1250b1b6])
I20260811 02:57:29.889055 16277 tablet_loader.cc:96] loaded metadata for tablet 420ed2b9216647498d9a2c956475ce4c (table default.0_4 [id=1dd6d5be9a9f4a6ba48fee38b5e87d63])
I20260811 02:57:29.889377 16277 tablet_loader.cc:96] loaded metadata for tablet a4ff2780d1b54a829f5a65f9b36d10ec (table default.0_1 [id=6c64c679309b4f29ab1ba582a3bcff07])
I20260811 02:57:29.889746 16277 tablet_loader.cc:96] loaded metadata for tablet e55d01af74a84dc391101460f2a9c063 (table default.1_2 [id=acb767fa16b7453da75b503e3c4be4f5])
I20260811 02:57:29.890091 16277 tablet_loader.cc:96] loaded metadata for tablet ee6c637392954d28abe1172c900498b2 (table default.1_1 [id=2b6d1a80a21d49ec8120aeac1c9b2764])
I20260811 02:57:29.890372 16277 tablet_loader.cc:96] loaded metadata for tablet f2e8bb100c3f41d5b74490a795fe638e (table default.1_4 [id=1765740dfbf5417e90b87056985f59e1])
I20260811 02:57:29.890658 16277 tablet_loader.cc:96] loaded metadata for tablet f78e24d030f549fb86183de598e3576b (table test-workload [id=f0728c3f7ecc49b5ae49a5bd89ce2df8])
I20260811 02:57:29.890966 16277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:29.892469 16277 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:29.892582 16277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:29.894707 16277 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:29.896584 16277 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Loaded TSK: 0
I20260811 02:57:29.897843 16277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:29.925457 15926 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50888:
name: "default.0_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:29.931228 15927 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50894:
name: "default.1_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:57:30.000545 16250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:30.000946 16250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:30.001096 16250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:30.009866 16250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:30.010169 16250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:57:30.020455 16250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:37837
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=34685
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:30.022184 16250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:30.024189 16250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:30.040023 16283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:30.040023 16285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:30.040934 16282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:30.041522 16250 server_base.cc:1034] running on GCE node
I20260811 02:57:30.042282 16250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:30.043394 16250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:30.044657 16250 hybrid_clock.cc:648] HybridClock initialized: now 1786417050044588 us; error 54 us; skew 500 ppm
I20260811 02:57:30.045015 16250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:30.048029 16250 webserver.cc:460] Webserver started at http://127.13.94.129:34685/ using document root <none> and password file <none>
I20260811 02:57:30.048841 16250 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:30.048987 16250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:30.055025 16250 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 02:57:30.061018 16292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:30.062758 16250 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260811 02:57:30.062963 16250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "42a22db0f41a4489a0dc8ce5315e4428"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:30.064095 16250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:30.094867 16250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:30.095793 16250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:30.096118 16250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:30.097376 16250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:30.099906 16299 ts_tablet_manager.cc:542] Loading tablet metadata (0/9 complete)
I20260811 02:57:30.124078 16250 ts_tablet_manager.cc:579] Loaded tablet metadata (9 total tablets, 9 live tablets)
I20260811 02:57:30.124220 16250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.025s user 0.002s sys 0.000s
I20260811 02:57:30.124329 16250 ts_tablet_manager.cc:594] Registering tablets (0/9 complete)
I20260811 02:57:30.128211 16299 tablet_bootstrap.cc:492] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:30.151621 16299 log.cc:824] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:30.153386 16250 ts_tablet_manager.cc:610] Registered 9 tablets
I20260811 02:57:30.153492 16250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.029s user 0.023s sys 0.003s
I20260811 02:57:30.156729 16299 tablet_bootstrap.cc:492] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:30.157508 16299 tablet_bootstrap.cc:492] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:30.159495 16299 ts_tablet_manager.cc:1397] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.032s user 0.017s sys 0.012s
I20260811 02:57:30.180393 16299 raft_consensus.cc:348] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.182534 16299 raft_consensus.cc:729] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.183650 16299 consensus_queue.cc:260] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.183997 16299 raft_consensus.cc:388] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:30.184196 16299 raft_consensus.cc:482] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:30.184433 16299 raft_consensus.cc:3037] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:30.192646 16299 raft_consensus.cc:504] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.193506 16299 leader_election.cc:302] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:30.194504 16299 leader_election.cc:288] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:30.194754 16397 raft_consensus.cc:2781] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:30.196918 16299 ts_tablet_manager.cc:1428] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.037s user 0.034s sys 0.003s
I20260811 02:57:30.197518 16299 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:30.201481 16397 raft_consensus.cc:686] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:30.202318 16397 consensus_queue.cc:237] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.216078 16250 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:37837
I20260811 02:57:30.216142 16424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:37837 every 8 connection(s)
I20260811 02:57:30.218102 16250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:30.219936 16299 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:30.220400 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 16250
I20260811 02:57:30.220636 16299 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:30.223198 16299 ts_tablet_manager.cc:1397] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.013s
I20260811 02:57:30.224524 16299 raft_consensus.cc:348] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.224776 16299 raft_consensus.cc:729] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.225055 16299 consensus_queue.cc:260] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.225281 16299 raft_consensus.cc:388] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:30.225418 16299 raft_consensus.cc:482] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:30.225558 16299 raft_consensus.cc:3037] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:30.229045 16299 raft_consensus.cc:504] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.229413 16299 leader_election.cc:302] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:30.229712 16299 leader_election.cc:288] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:57:30.229769 16397 raft_consensus.cc:2781] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:30.229992 16397 raft_consensus.cc:686] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:30.230319 16397 consensus_queue.cc:237] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.230484 16299 ts_tablet_manager.cc:1428] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 02:57:30.230822 16299 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:30.270660 16426 heartbeater.cc:347] Connected to a master server at 127.13.94.187:44947
I20260811 02:57:30.270947 16427 heartbeater.cc:347] Connected to a master server at 127.13.94.188:44361
I20260811 02:57:30.271021 16426 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:30.271142 16427 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:30.272213 16427 heartbeater.cc:511] Master 127.13.94.188:44361 requested a full tablet report, sending...
I20260811 02:57:30.275102 16432 heartbeater.cc:347] Connected to a master server at 127.13.94.189:45939
I20260811 02:57:30.275259 16426 heartbeater.cc:511] Master 127.13.94.187:44947 requested a full tablet report, sending...
I20260811 02:57:30.275271 16432 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:30.275933 16432 heartbeater.cc:511] Master 127.13.94.189:45939 requested a full tablet report, sending...
I20260811 02:57:30.278821 16207 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:30.278838 16114 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:30.278868 16019 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:30.283087 16124 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 3 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 2 index: 45. Preceding OpId from leader: term: 3 index: 46. (index mismatch)
I20260811 02:57:30.284796 16275 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
I20260811 02:57:30.286777 16428 heartbeater.cc:347] Connected to a master server at 127.13.94.190:38889
I20260811 02:57:30.286944 16428 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:30.287416 16428 heartbeater.cc:511] Master 127.13.94.190:38889 requested a full tablet report, sending...
I20260811 02:57:30.287959 16299 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:30.288730 16299 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:30.290493 16299 ts_tablet_manager.cc:1397] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.060s user 0.018s sys 0.029s
I20260811 02:57:30.290482 15927 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:30.291706 16299 raft_consensus.cc:348] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.292074 16299 raft_consensus.cc:729] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.292503 16299 consensus_queue.cc:260] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.292904 16299 raft_consensus.cc:388] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:30.293081 16299 raft_consensus.cc:482] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:30.293257 16299 raft_consensus.cc:3037] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:30.293555 15927 catalog_manager.cc:5495] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:30.294543 15927 catalog_manager.cc:5495] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:30.297430 16299 raft_consensus.cc:504] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.298401 16299 leader_election.cc:302] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:30.298977 16399 raft_consensus.cc:2781] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:30.299177 16399 raft_consensus.cc:686] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:30.299572 16399 consensus_queue.cc:237] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.304898 16217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 3 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 2 index: 45. Preceding OpId from leader: term: 3 index: 47. (index mismatch)
I20260811 02:57:30.306644 16275 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
I20260811 02:57:30.307497 16299 leader_election.cc:288] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:57:30.308879 16299 ts_tablet_manager.cc:1428] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.018s user 0.003s sys 0.004s
I20260811 02:57:30.309996 16299 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:30.313261 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:30.314175 16029 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 3 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 2 index: 45. Preceding OpId from leader: term: 3 index: 47. (index mismatch)
I20260811 02:57:30.315155 16275 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
I20260811 02:57:30.327456 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:30.328780 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.186:40871
--webserver_interface=127.13.94.186
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871 with env {}
I20260811 02:57:30.332479 16275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:30.332852 16275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:30.333209 16445 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:30.333701 16445 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:30.345691 16450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:30.346184 16450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:30.350615 16450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:30.350939 16450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:30.358103 16299 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:30.358839 16299 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:30.360400 16299 ts_tablet_manager.cc:1397] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.051s user 0.022s sys 0.012s
I20260811 02:57:30.361505 16299 raft_consensus.cc:348] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.361755 16299 raft_consensus.cc:729] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.362092 16299 consensus_queue.cc:260] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.362335 16299 raft_consensus.cc:388] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:30.362484 16299 raft_consensus.cc:482] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:30.362601 16299 raft_consensus.cc:3037] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:30.362673 16275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:30.362994 16275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:30.366276 16299 raft_consensus.cc:504] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.366603 16299 leader_election.cc:302] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:30.366875 16399 raft_consensus.cc:2781] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:30.367025 16399 raft_consensus.cc:686] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:30.367240 16399 consensus_queue.cc:237] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.370002 16299 leader_election.cc:288] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:57:30.370543 16299 ts_tablet_manager.cc:1428] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 02:57:30.370844 16299 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:30.373572 15927 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.94.129:57245
I20260811 02:57:30.378733 16428 heartbeater.cc:503] Master 127.13.94.190:38889 was elected leader, sending a full tablet report...
I20260811 02:57:30.382074 16445 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:30.382560 16445 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:30.382925 15927 catalog_manager.cc:5495] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:30.383701 15927 catalog_manager.cc:5495] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:30.391270 16299 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:30.391856 16299 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:30.393118 16299 ts_tablet_manager.cc:1397] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.003s
I20260811 02:57:30.394183 16299 raft_consensus.cc:348] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.394456 16299 raft_consensus.cc:729] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.394727 16299 consensus_queue.cc:260] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.394976 16299 raft_consensus.cc:388] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:30.395097 16299 raft_consensus.cc:482] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:30.395211 16299 raft_consensus.cc:3037] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:30.400671 16299 raft_consensus.cc:504] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.400631 16275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:30.400995 16275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:30.401122 16299 leader_election.cc:302] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:30.401439 16399 raft_consensus.cc:2781] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:30.401595 16399 raft_consensus.cc:686] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:30.401878 16399 consensus_queue.cc:237] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.401530 16451 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:30.402110 16451 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:30.403174 16299 leader_election.cc:288] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:57:30.403759 16299 ts_tablet_manager.cc:1428] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.010s user 0.007s sys 0.003s
I20260811 02:57:30.404057 16299 tablet_bootstrap.cc:492] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:30.406468 15927 catalog_manager.cc:5495] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:30.423573 16299 tablet_bootstrap.cc:492] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:30.424147 16299 tablet_bootstrap.cc:492] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:30.425437 16299 ts_tablet_manager.cc:1397] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.021s user 0.016s sys 0.002s
I20260811 02:57:30.426286 16299 raft_consensus.cc:348] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.426482 16299 raft_consensus.cc:729] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.426862 16299 consensus_queue.cc:260] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.427062 16299 raft_consensus.cc:388] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:30.427146 16299 raft_consensus.cc:482] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:30.427228 16299 raft_consensus.cc:3037] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:30.430294 16299 raft_consensus.cc:504] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.430590 16299 leader_election.cc:302] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:30.430866 16399 raft_consensus.cc:2781] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:30.431025 16399 raft_consensus.cc:686] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:30.431282 16399 consensus_queue.cc:237] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.437770 16299 leader_election.cc:288] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:30.438838 16299 ts_tablet_manager.cc:1428] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.013s user 0.004s sys 0.004s
I20260811 02:57:30.439162 16299 tablet_bootstrap.cc:492] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:30.442382 16451 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } }
I20260811 02:57:30.442706 16451 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:30.451593 15927 catalog_manager.cc:5495] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:30.470340 16299 tablet_bootstrap.cc:492] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:30.470917 16299 tablet_bootstrap.cc:492] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:30.472031 16299 ts_tablet_manager.cc:1397] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.014s
I20260811 02:57:30.472800 16299 raft_consensus.cc:348] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.473011 16299 raft_consensus.cc:729] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.473340 16299 consensus_queue.cc:260] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.473549 16299 raft_consensus.cc:388] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:30.473717 16299 raft_consensus.cc:482] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:30.473857 16299 raft_consensus.cc:3037] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:30.476186 16299 raft_consensus.cc:504] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.476501 16299 leader_election.cc:302] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:30.476760 16299 leader_election.cc:288] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:30.476852 16399 raft_consensus.cc:2781] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:30.477034 16399 raft_consensus.cc:686] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:30.477291 16299 ts_tablet_manager.cc:1428] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260811 02:57:30.477293 16399 consensus_queue.cc:237] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.477607 16299 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:30.483592 15927 catalog_manager.cc:5495] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:30.516000 16299 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:30.516646 16299 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:30.517966 16299 ts_tablet_manager.cc:1397] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.040s user 0.020s sys 0.015s
I20260811 02:57:30.518721 16299 raft_consensus.cc:348] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.518916 16299 raft_consensus.cc:729] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.519209 16299 consensus_queue.cc:260] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.519393 16299 raft_consensus.cc:388] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:30.519556 16299 raft_consensus.cc:482] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:30.519686 16299 raft_consensus.cc:3037] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:30.522015 16299 raft_consensus.cc:504] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.522300 16299 leader_election.cc:302] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:30.522612 16299 leader_election.cc:288] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:57:30.522681 16399 raft_consensus.cc:2781] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:30.522871 16399 raft_consensus.cc:686] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:30.523208 16299 ts_tablet_manager.cc:1428] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260811 02:57:30.523123 16399 consensus_queue.cc:237] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.523550 16299 tablet_bootstrap.cc:492] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:30.540721 15927 catalog_manager.cc:5495] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:30.545179 16299 tablet_bootstrap.cc:492] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:30.545845 16299 tablet_bootstrap.cc:492] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:30.547906 16299 ts_tablet_manager.cc:1397] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.019s
I20260811 02:57:30.548848 16299 raft_consensus.cc:348] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.549170 16299 raft_consensus.cc:729] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.549500 16299 consensus_queue.cc:260] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.549747 16299 raft_consensus.cc:388] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:30.549875 16299 raft_consensus.cc:482] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:30.549968 16299 raft_consensus.cc:3037] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:30.553076 16299 raft_consensus.cc:504] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.553352 16299 leader_election.cc:302] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:30.553661 16399 raft_consensus.cc:2781] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:30.553892 16399 raft_consensus.cc:686] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:30.554157 16399 consensus_queue.cc:237] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:30.560510 16299 leader_election.cc:288] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:30.562448 16299 ts_tablet_manager.cc:1428] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.014s user 0.005s sys 0.001s
I20260811 02:57:30.570240 15927 catalog_manager.cc:5495] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
W20260811 02:57:30.839982 16457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:30.840322 16457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:30.840531 16457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:30.850112 16457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:30.850250 16457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:30.850358 16457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:30.850455 16457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:30.861495 16457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.186:40871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/info.pb
--webserver_interface=127.13.94.186
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:30.863158 16457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:30.865360 16457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:30.878917 16487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:30.879016 16485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:30.878973 16484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:30.880266 16457 server_base.cc:1034] running on GCE node
I20260811 02:57:30.881306 16457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:30.883081 16457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:30.884346 16457 hybrid_clock.cc:648] HybridClock initialized: now 1786417050884275 us; error 60 us; skew 500 ppm
I20260811 02:57:30.884702 16457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:30.887339 16457 webserver.cc:460] Webserver started at http://127.13.94.186:34273/ using document root <none> and password file <none>
I20260811 02:57:30.888121 16457 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:30.888283 16457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:30.888643 16457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:30.891688 16457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/instance:
uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-p3t5"
I20260811 02:57:30.892570 16457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal/instance:
uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-p3t5"
I20260811 02:57:30.898825 16457 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 02:57:30.903353 16493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:30.905021 16457 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:57:30.905241 16457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-p3t5"
I20260811 02:57:30.905503 16457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:30.952013 16457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:30.953523 16457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:30.958312 16457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:31.004668 15927 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50962:
name: "default.1_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:31.006037 15926 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50952:
name: "default.0_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:31.073235 16348 tablet_service.cc:1467] Processing CreateTablet for tablet 4694adb271df417297bb20aed3fbe6ac (DEFAULT_TABLE table=default.0_7 [id=ce943c1496344fab928bc054e7206385]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:31.073287 16349 tablet_service.cc:1467] Processing CreateTablet for tablet a65dd69fcfaa429697f56a4329c97145 (DEFAULT_TABLE table=default.1_7 [id=d1f9862292724158acf991e7f1aa1f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:31.074205 16348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4694adb271df417297bb20aed3fbe6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:31.074580 16349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a65dd69fcfaa429697f56a4329c97145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:31.084863 16573 tablet_bootstrap.cc:492] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:31.087638 16573 tablet_bootstrap.cc:654] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:31.090236 16573 tablet_bootstrap.cc:492] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:31.090492 16573 ts_tablet_manager.cc:1397] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:57:31.090662 16457 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.186:40871
I20260811 02:57:31.090801 16579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.186:40871 every 8 connection(s)
I20260811 02:57:31.091449 16573 raft_consensus.cc:348] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:31.091626 16573 raft_consensus.cc:374] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:31.091753 16573 raft_consensus.cc:729] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:31.092010 16573 consensus_queue.cc:260] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:31.092198 16573 raft_consensus.cc:388] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:31.092308 16573 raft_consensus.cc:482] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:31.092439 16573 raft_consensus.cc:3037] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.092563 16457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/info.pb
I20260811 02:57:31.094974 16573 raft_consensus.cc:504] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:31.095291 16573 leader_election.cc:302] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:31.095731 16573 leader_election.cc:288] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:31.095924 16582 raft_consensus.cc:2781] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:31.097108 16573 ts_tablet_manager.cc:1428] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:57:31.097432 16573 tablet_bootstrap.cc:492] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:31.097528 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 16457
I20260811 02:57:31.098790 16580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:31.099299 16573 tablet_bootstrap.cc:654] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:31.109264 16582 raft_consensus.cc:686] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:31.109735 16582 consensus_queue.cc:237] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:31.114182 16580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:31.118176 16573 tablet_bootstrap.cc:492] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:31.118489 16573 ts_tablet_manager.cc:1397] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.021s user 0.017s sys 0.000s
I20260811 02:57:31.119473 16573 raft_consensus.cc:348] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:31.119793 16573 raft_consensus.cc:374] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:31.119946 16573 raft_consensus.cc:729] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:31.120254 16573 consensus_queue.cc:260] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:31.120573 16573 raft_consensus.cc:388] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:31.120739 16573 raft_consensus.cc:482] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:31.120898 16573 raft_consensus.cc:3037] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.123919 16573 raft_consensus.cc:504] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:31.124264 16573 leader_election.cc:302] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:31.124529 16573 leader_election.cc:288] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:31.125082 16573 ts_tablet_manager.cc:1428] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:57:31.125219 16582 raft_consensus.cc:2781] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:31.125455 16582 raft_consensus.cc:686] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:31.125728 16582 consensus_queue.cc:237] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:31.128223 15925 catalog_manager.cc:5495] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:31.147696 15925 catalog_manager.cc:5495] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:31.155529 16580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:31.164448 16580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:31.171736 16580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:31.179952 16580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:31.194676 16580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Bootstrap starting.
I20260811 02:57:31.198907 16580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:31.200445 16580 log.cc:824] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:31.203960 16580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: No bootstrap required, opened a new log
I20260811 02:57:31.212185 16580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } }
I20260811 02:57:31.212587 16580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:31.212673 16580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213a74fcc5cd4d3f9be2f848f3b3fd49, State: Initialized, Role: FOLLOWER
I20260811 02:57:31.213236 16580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } }
I20260811 02:57:31.214255 16593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } } }
I20260811 02:57:31.214695 16593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 02:57:31.215478 16520 tablet.cc:2369] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:31.215546 16580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:31.233037 16605 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:31.233228 16605 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:31.238615 16457 master_runner.cc:428] Detected that this master 213a74fcc5cd4d3f9be2f848f3b3fd49 is joining an existing cluster
I20260811 02:57:31.238878 16457 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.94.186:40871
I20260811 02:57:31.313866 15927 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.94.186:40871: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49"
member_type: NON_VOTER
last_known_addr {
host: "127.13.94.186"
port: 40871
}
attrs {
promote: true
}
}
dest_uuid: "92aef555682a43a1a1619b2bc9ba7ce1"
cas_config_opid_index: 37
I20260811 02:57:31.315528 15927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 59, Committed index: 59, Last appended: 3.59, Last appended by leader: 45, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } }
I20260811 02:57:31.318977 16124 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 3 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 3 index: 59. Preceding OpId from leader: term: 3 index: 60. (index mismatch)
I20260811 02:57:31.319142 16551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 0 FOLLOWER]: Advancing to term 3
I20260811 02:57:31.319295 16217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 3 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 3 index: 59. Preceding OpId from leader: term: 3 index: 60. (index mismatch)
I20260811 02:57:31.320079 16474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 59, Time since last communication: 0.000s
I20260811 02:57:31.320602 16473 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 59, Time since last communication: 0.000s
I20260811 02:57:31.322963 16028 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 3 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 3 index: 59. Preceding OpId from leader: term: 3 index: 60. (index mismatch)
I20260811 02:57:31.323398 16551 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 60. (index mismatch)
I20260811 02:57:31.323818 16474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 59, Time since last communication: 0.000s
I20260811 02:57:31.324301 16473 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:31.326709 16271 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 LEADER]: Committing config change with OpId 3.60: config changed from index 37 to 60, NON_VOTER 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186) added. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.327508 16028 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 3 FOLLOWER]: Committing config change with OpId 3.60: config changed from index 37 to 60, NON_VOTER 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186) added. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.331499 16271 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.94.186:40871
I20260811 02:57:31.330899 16217 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 3 FOLLOWER]: Committing config change with OpId 3.60: config changed from index 37 to 60, NON_VOTER 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186) added. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.332118 16453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.332593 16453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:31.333287 16529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.333737 16529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:31.333994 16529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.334252 16529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:31.334098 16271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.334496 16271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:31.334625 16124 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 3 FOLLOWER]: Committing config change with OpId 3.60: config changed from index 37 to 60, NON_VOTER 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186) added. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.341892 16457 master.cc:537] Master@127.13.94.186:40871 shutting down...
I20260811 02:57:31.345597 16617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.346040 16617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:31.346661 16619 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.347289 16619 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:31.351127 16618 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.351740 16618 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:31.352862 16551 raft_consensus.cc:858] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } }.
Pending config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } }:
New config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
W20260811 02:57:31.353087 16551 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 18' and following 42 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } }.
Pending config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } }:
New config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } }
I20260811 02:57:31.362739 16551 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 NON_PARTICIPANT]: Committing config change with OpId 1.15: config changed from index -1 to 15, NON_VOTER e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:31.369715 16622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:31.370164 16622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:57:31.370713 16593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:31.370958 16593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:57:31.368746 16626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: true } } }
I20260811 02:57:31.371604 16626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:57:31.372874 16620 mvcc.cc:205] Tried to move back new op lower bound from 7317164202289180672 to 7317164194469126144. Current Snapshot: MvccSnapshot[applied={T|T < 7317164202240200704}]
I20260811 02:57:31.375900 16457 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 02:57:31.376430 16457 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 02:57:31.376649 16457 tablet_replica.cc:330] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: stopping tablet replica
I20260811 02:57:31.377460 16473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.377864 16473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
W20260811 02:57:31.379138 15911 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 02:57:31.381045 15911 consensus_peers.cc:487] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 -> Peer 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186:40871): Couldn't send request to peer 213a74fcc5cd4d3f9be2f848f3b3fd49. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 02:57:31.385226 16457 master.cc:552] Master@127.13.94.186:40871 shutdown complete.
I20260811 02:57:31.385317 16457 master_runner.cc:305] Clearing existing system tablet
I20260811 02:57:31.385677 16457 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:31.385794 16457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:31.388135 16457 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 02:57:31.389734 16632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:31.390116 16457 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:31.390383 16457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 02:57:31.390491 16457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-p3t5"
I20260811 02:57:31.393684 16457 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:31.398461 16457 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 02:57:31.398641 16457 log.cc:1196] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Deleting WAL directory at /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal/wals/00000000000000000000000000000000
I20260811 02:57:31.399175 16457 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Deleting consensus metadata
I20260811 02:57:31.399451 16457 master_runner.cc:315] Copying system tablet from 127.13.94.190:38889
I20260811 02:57:31.402551 16457 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: tablet copy: Beginning tablet copy session from remote peer at address 127.13.94.190:38889
I20260811 02:57:31.410847 15948 tablet_copy_service.cc:140] P 92aef555682a43a1a1619b2bc9ba7ce1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 213a74fcc5cd4d3f9be2f848f3b3fd49 ({username='slave'} at 127.0.0.1:50992)
I20260811 02:57:31.411206 15948 tablet_copy_service.cc:161] P 92aef555682a43a1a1619b2bc9ba7ce1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 213a74fcc5cd4d3f9be2f848f3b3fd49 at {username='slave'} at 127.0.0.1:50992: session id = 213a74fcc5cd4d3f9be2f848f3b3fd49-00000000000000000000000000000000
I20260811 02:57:31.414448 15948 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 02:57:31.416810 16457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 02:57:31.421802 16457 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: tablet copy: Starting download of 0 data blocks...
I20260811 02:57:31.422129 16457 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: tablet copy: Starting download of 1 WAL segments...
I20260811 02:57:31.425050 16457 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 02:57:31.428442 15948 tablet_copy_service.cc:342] P 92aef555682a43a1a1619b2bc9ba7ce1: Request end of tablet copy session 213a74fcc5cd4d3f9be2f848f3b3fd49-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:50992
I20260811 02:57:31.428654 15948 tablet_copy_service.cc:433] P 92aef555682a43a1a1619b2bc9ba7ce1: ending tablet copy session 213a74fcc5cd4d3f9be2f848f3b3fd49-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 213a74fcc5cd4d3f9be2f848f3b3fd49
I20260811 02:57:31.439847 16457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:31.442864 16639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:31.452768 16640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:31.456007 15927 master.cc:648] Network error: unable to get registration information for peer 213a74fcc5cd4d3f9be2f848f3b3fd49: Client connection negotiation failed: client connection to 127.13.94.186:40871: connect: Connection refused (error 111)
W20260811 02:57:31.457154 16644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:31.458182 16457 server_base.cc:1034] running on GCE node
I20260811 02:57:31.458559 16457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:31.459360 16457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:31.460582 16457 hybrid_clock.cc:648] HybridClock initialized: now 1786417051460531 us; error 35 us; skew 500 ppm
I20260811 02:57:31.460848 16457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:31.462493 16457 webserver.cc:460] Webserver started at http://127.13.94.186:34211/ using document root <none> and password file <none>
I20260811 02:57:31.462852 16457 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:31.462937 16457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:31.466212 16457 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260811 02:57:31.468245 16649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:31.468819 16457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 02:57:31.468950 16457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-p3t5"
I20260811 02:57:31.469121 16457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:31.495612 16457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:31.496542 16457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:31.519222 16706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.186:40871 every 8 connection(s)
I20260811 02:57:31.519222 16457 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.186:40871
I20260811 02:57:31.520699 16457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/info.pb
I20260811 02:57:31.526286 16707 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:31.530444 16707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Bootstrap starting.
I20260811 02:57:31.591327 16707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Bootstrap replayed 1/1 log segments. Stats: ops{read=60 overwritten=0 applied=60 ignored=0} inserts{seen=25 ignored=0} mutations{seen=40 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:31.591881 16707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Bootstrap complete.
I20260811 02:57:31.592933 16707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } }
I20260811 02:57:31.593238 16707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 213a74fcc5cd4d3f9be2f848f3b3fd49, State: Initialized, Role: LEARNER
I20260811 02:57:31.593525 16707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 60, Last appended: 3.60, Last appended by leader: 60, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } }
I20260811 02:57:31.594527 16710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.594844 16707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:31.594961 16710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:31.603861 16722 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:31.603999 16722 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: loading cluster ID for follower catalog manager: success
I20260811 02:57:31.607187 16722 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: acquiring CA information for follower catalog manager: success
I20260811 02:57:31.609489 16722 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:31.788295 16473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.788710 16473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:31.790278 16679 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 LEARNER]: Deduplicated request from leader. Original: 1.17->[1.18-3.60] Dedup: 3.60->[]
I20260811 02:57:31.790853 16710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: NON_VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: true } } }
I20260811 02:57:31.791236 16710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:31.791702 16474 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: attempting to promote NON_VOTER 213a74fcc5cd4d3f9be2f848f3b3fd49 to VOTER
I20260811 02:57:31.792886 16474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 60, Committed index: 60, Last appended: 3.60, Last appended by leader: 45, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:31.795184 16679 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 LEARNER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 3 index: 60. Preceding OpId from leader: term: 3 index: 61. (index mismatch)
I20260811 02:57:31.795282 16124 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 3 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 3 index: 60. Preceding OpId from leader: term: 3 index: 61. (index mismatch)
I20260811 02:57:31.795329 16217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 3 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 3 index: 60. Preceding OpId from leader: term: 3 index: 61. (index mismatch)
I20260811 02:57:31.795447 16028 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 3 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 3 index: 60. Preceding OpId from leader: term: 3 index: 61. (index mismatch)
I20260811 02:57:31.796056 16473 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20260811 02:57:31.796315 16453 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20260811 02:57:31.796571 16271 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20260811 02:57:31.797173 16474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20260811 02:57:31.799556 16271 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 LEADER]: Committing config change with OpId 3.61: config changed from index 60 to 61, 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.801090 16124 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 3 FOLLOWER]: Committing config change with OpId 3.61: config changed from index 60 to 61, 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.801172 16217 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 3 FOLLOWER]: Committing config change with OpId 3.61: config changed from index 60 to 61, 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.801506 16679 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 FOLLOWER]: Committing config change with OpId 3.61: config changed from index 60 to 61, 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.804498 16028 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 3 FOLLOWER]: Committing config change with OpId 3.61: config changed from index 60 to 61, 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.805081 16617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.805392 16617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:31.805284 16618 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.805579 16618 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:31.806581 16710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.806931 16710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:31.807498 16453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.807626 16473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.807533 16529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.808018 16453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:31.808388 16619 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.808712 16619 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:31.808780 16473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:31.809094 16529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:31.808881 16453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.809492 16453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:31.809403 16271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:31.809780 16271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:32.192775 15927 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51008:
name: "default.0_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:32.216688 15925 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51022:
name: "default.1_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:32.260092 16349 tablet_service.cc:1467] Processing CreateTablet for tablet 13b5598d42214d908fe5b77e009b8939 (DEFAULT_TABLE table=default.0_8 [id=5ac90fca3b9d4650b0f568eef6113738]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:32.260349 16348 tablet_service.cc:1467] Processing CreateTablet for tablet a6142c6b82194a798989a9d808341f60 (DEFAULT_TABLE table=default.1_8 [id=1b031c8533b1417591024c3b6a9b2b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:32.260674 16349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13b5598d42214d908fe5b77e009b8939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:32.260951 16348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6142c6b82194a798989a9d808341f60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:32.269484 16760 tablet_bootstrap.cc:492] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:32.271291 16760 tablet_bootstrap.cc:654] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:32.275049 16760 tablet_bootstrap.cc:492] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:32.275229 16760 ts_tablet_manager.cc:1397] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 02:57:32.276084 16760 raft_consensus.cc:348] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:32.276299 16760 raft_consensus.cc:374] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:32.276410 16760 raft_consensus.cc:729] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:32.276643 16760 consensus_queue.cc:260] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:32.276813 16760 raft_consensus.cc:388] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:32.276918 16760 raft_consensus.cc:482] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:32.277058 16760 raft_consensus.cc:3037] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:32.279726 16760 raft_consensus.cc:504] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:32.280023 16760 leader_election.cc:302] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:32.280699 16760 leader_election.cc:288] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:32.280828 16764 raft_consensus.cc:2781] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:32.283090 16764 raft_consensus.cc:686] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:32.283685 16760 ts_tablet_manager.cc:1428] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.008s user 0.005s sys 0.004s
I20260811 02:57:32.283659 16764 consensus_queue.cc:237] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:32.283984 16760 tablet_bootstrap.cc:492] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:32.285651 16760 tablet_bootstrap.cc:654] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:32.288100 16760 tablet_bootstrap.cc:492] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428: No bootstrap required, opened a new log
I20260811 02:57:32.288293 16760 ts_tablet_manager.cc:1397] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260811 02:57:32.289136 16760 raft_consensus.cc:348] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:32.289357 16760 raft_consensus.cc:374] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:32.289448 16760 raft_consensus.cc:729] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:32.289846 16760 consensus_queue.cc:260] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:32.290068 16760 raft_consensus.cc:388] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:32.290273 16760 raft_consensus.cc:482] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:32.290433 16760 raft_consensus.cc:3037] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:32.291391 15925 catalog_manager.cc:5495] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:32.293716 16760 raft_consensus.cc:504] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:32.294001 16760 leader_election.cc:302] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:32.294231 16760 leader_election.cc:288] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:32.294472 16765 raft_consensus.cc:2781] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:32.294778 16760 ts_tablet_manager.cc:1428] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:57:32.306651 16765 raft_consensus.cc:686] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:32.307080 16765 consensus_queue.cc:237] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:32.311007 15925 catalog_manager.cc:5495] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:32.399926 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 16250
I20260811 02:57:32.408388 16420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:32.521641 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 16250
I20260811 02:57:32.552840 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 15877
I20260811 02:57:32.554204 15960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:32.683221 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 15877
I20260811 02:57:32.711170 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 15968
I20260811 02:57:32.712422 16054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:32.827521 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 15968
I20260811 02:57:32.855342 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 16061
I20260811 02:57:32.856581 16146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:32.972852 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 16061
I20260811 02:57:32.999815 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 16154
I20260811 02:57:33.000972 16241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:33.119065 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 16154
I20260811 02:57:33.143253 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 16457
I20260811 02:57:33.144526 16702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:33.259271 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 16457
I20260811 02:57:33.285550 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:38889
--webserver_interface=127.13.94.190
--webserver_port=43213
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871 with env {}
W20260811 02:57:33.676798 16784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:33.677217 16784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:33.677367 16784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:33.685060 16784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:33.685225 16784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:33.685324 16784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:33.685427 16784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:33.695405 16784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=43213
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:33.697214 16784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:33.699693 16784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:33.712550 16808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:33.712528 16806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:33.712678 16805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:33.713531 16784 server_base.cc:1034] running on GCE node
I20260811 02:57:33.714426 16784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:33.715899 16784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:33.717185 16784 hybrid_clock.cc:648] HybridClock initialized: now 1786417053717094 us; error 68 us; skew 500 ppm
I20260811 02:57:33.717576 16784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:33.726400 16784 webserver.cc:460] Webserver started at http://127.13.94.190:43213/ using document root <none> and password file <none>
I20260811 02:57:33.727283 16784 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:33.727437 16784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:33.733156 16784 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:57:33.736308 16814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:33.737879 16784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:57:33.738098 16784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "92aef555682a43a1a1619b2bc9ba7ce1"
format_stamp: "Formatted at 2026-08-11 02:57:18 on dist-test-slave-p3t5"
I20260811 02:57:33.739132 16784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:33.750672 16784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:33.751554 16784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:33.751865 16784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:33.772300 16784 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:38889
I20260811 02:57:33.772329 16874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:38889 every 8 connection(s)
I20260811 02:57:33.774039 16784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:33.780057 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 16784
I20260811 02:57:33.780800 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:45939
--webserver_interface=127.13.94.189
--webserver_port=39639
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871 with env {}
I20260811 02:57:33.781906 16875 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:33.788028 16875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Bootstrap starting.
I20260811 02:57:33.831875 16875 log.cc:824] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:33.896466 16875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Bootstrap replayed 1/1 log segments. Stats: ops{read=66 overwritten=0 applied=66 ignored=0} inserts{seen=29 ignored=0} mutations{seen=45 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:33.897152 16875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Bootstrap complete.
I20260811 02:57:33.906148 16875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:33.907845 16875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92aef555682a43a1a1619b2bc9ba7ce1, State: Initialized, Role: FOLLOWER
I20260811 02:57:33.909241 16875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 66, Last appended: 3.66, Last appended by leader: 66, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:33.910462 16881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:33.911135 16881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:33.911440 16875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:33.924404 16895 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:33.924741 16895 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: loading cluster ID for follower catalog manager: success
I20260811 02:57:33.930040 16895 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: acquiring CA information for follower catalog manager: success
I20260811 02:57:33.934171 16895 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:33.933051 16784 master_runner.cc:186] Error getting master registration for 127.13.94.189:45939: Network error: Client connection negotiation failed: client connection to 127.13.94.189:45939: connect: Connection refused (error 111),
I20260811 02:57:33.941758 16784 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:57:34.226249 16877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:34.226686 16877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:34.226866 16877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:34.234573 16877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:34.234723 16877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:34.234831 16877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:34.234941 16877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:34.245790 16877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:45939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--webserver_port=39639
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:34.247324 16877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:34.249396 16877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:34.266350 16902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:34.267314 16903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:34.268659 16905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:34.268836 16877 server_base.cc:1034] running on GCE node
I20260811 02:57:34.270236 16877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:34.272610 16877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:34.273924 16877 hybrid_clock.cc:648] HybridClock initialized: now 1786417054273825 us; error 69 us; skew 500 ppm
I20260811 02:57:34.274449 16877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:34.278025 16877 webserver.cc:460] Webserver started at http://127.13.94.189:39639/ using document root <none> and password file <none>
I20260811 02:57:34.279150 16877 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:34.279335 16877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:34.287900 16877 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 02:57:34.292232 16911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:34.294353 16877 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:57:34.294600 16877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "d72eb236aa6043eb91a5dfce161d5057"
format_stamp: "Formatted at 2026-08-11 02:57:19 on dist-test-slave-p3t5"
I20260811 02:57:34.296185 16877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:34.322182 16877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:34.323477 16877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:34.323859 16877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:34.350908 16966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:45939 every 8 connection(s)
I20260811 02:57:34.350946 16877 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:45939
I20260811 02:57:34.353263 16877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:57:34.353917 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 16877
I20260811 02:57:34.354689 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:44361
--webserver_interface=127.13.94.188
--webserver_port=36707
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871 with env {}
I20260811 02:57:34.364455 16967 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:34.371583 16967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Bootstrap starting.
I20260811 02:57:34.409348 16967 log.cc:824] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:34.482254 16967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Bootstrap replayed 1/1 log segments. Stats: ops{read=66 overwritten=0 applied=66 ignored=0} inserts{seen=29 ignored=0} mutations{seen=45 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:34.483275 16967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: Bootstrap complete.
I20260811 02:57:34.493184 16967 raft_consensus.cc:348] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:34.494436 16967 raft_consensus.cc:729] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72eb236aa6043eb91a5dfce161d5057, State: Initialized, Role: FOLLOWER
I20260811 02:57:34.495137 16967 consensus_queue.cc:260] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 66, Last appended: 3.66, Last appended by leader: 66, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:34.496207 16975 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:34.496770 16975 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:34.497342 16967 sys_catalog.cc:564] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:34.525035 16986 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:34.525170 16986 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: loading cluster ID for follower catalog manager: success
I20260811 02:57:34.533243 16986 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: acquiring CA information for follower catalog manager: success
I20260811 02:57:34.536639 16986 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:34.551558 16877 master_runner.cc:186] Error getting master registration for 127.13.94.188:44361: Network error: Client connection negotiation failed: client connection to 127.13.94.188:44361: connect: Connection refused (error 111),
I20260811 02:57:34.561470 16877 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:57:34.776185 16969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:34.776597 16969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:34.776732 16969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:34.788635 16969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:34.788776 16969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:34.788838 16969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:34.788934 16969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:34.800552 16969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:44361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.94.188
--webserver_port=36707
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:34.802367 16969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:34.804486 16969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:34.818152 16994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:34.818130 16995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:34.818936 16997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:34.823719 16969 server_base.cc:1034] running on GCE node
I20260811 02:57:34.824754 16969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:34.826659 16969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:34.827950 16969 hybrid_clock.cc:648] HybridClock initialized: now 1786417054827873 us; error 83 us; skew 500 ppm
I20260811 02:57:34.828408 16969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:34.831660 16969 webserver.cc:460] Webserver started at http://127.13.94.188:36707/ using document root <none> and password file <none>
I20260811 02:57:34.832450 16969 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:34.832597 16969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:34.839216 16969 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 02:57:34.843034 17005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:34.845160 16969 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:57:34.845386 16969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
uuid: "e3aa822be3af4af4a40b453c83d7d425"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:34.846733 16969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:34.872354 16969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:34.873314 16969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:34.873708 16969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:34.899596 16969 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.188:44361
I20260811 02:57:34.899607 17067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.188:44361 every 8 connection(s)
I20260811 02:57:34.901594 16969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
I20260811 02:57:34.909672 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 16969
I20260811 02:57:34.910279 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.187:44947
--webserver_interface=127.13.94.187
--webserver_port=35587
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871 with env {}
I20260811 02:57:34.911543 17068 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:34.917084 17068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap starting.
I20260811 02:57:34.947682 17068 log.cc:824] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:35.019111 17068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap replayed 1/1 log segments. Stats: ops{read=66 overwritten=0 applied=66 ignored=0} inserts{seen=29 ignored=0} mutations{seen=45 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:35.019876 17068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: Bootstrap complete.
I20260811 02:57:35.032236 17068 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:35.034096 17068 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3aa822be3af4af4a40b453c83d7d425, State: Initialized, Role: FOLLOWER
I20260811 02:57:35.035044 17068 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 66, Last appended: 3.66, Last appended by leader: 66, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:35.036257 17074 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:35.036775 17074 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:35.037508 17068 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:35.055713 17086 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:35.055891 17086 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: loading cluster ID for follower catalog manager: success
I20260811 02:57:35.063864 17086 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: acquiring CA information for follower catalog manager: success
I20260811 02:57:35.067001 17086 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:35.105821 16969 master_runner.cc:186] Error getting master registration for 127.13.94.187:44947: Network error: Client connection negotiation failed: client connection to 127.13.94.187:44947: connect: Connection refused (error 111),
I20260811 02:57:35.116955 16969 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:57:35.395592 17070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:35.395929 17070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:35.396064 17070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:35.403937 17070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:35.404103 17070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:35.404207 17070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:35.404317 17070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:35.415880 17070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.187:44947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.13.94.187
--webserver_port=35587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:35.417870 17070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:35.420570 17070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:35.434865 17096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:35.435130 17095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:35.435494 17100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:35.435945 17070 server_base.cc:1034] running on GCE node
I20260811 02:57:35.437098 17070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:35.439117 17070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:35.440356 17070 hybrid_clock.cc:648] HybridClock initialized: now 1786417055440260 us; error 69 us; skew 500 ppm
I20260811 02:57:35.440779 17070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:35.443715 17070 webserver.cc:460] Webserver started at http://127.13.94.187:35587/ using document root <none> and password file <none>
I20260811 02:57:35.444511 17070 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:35.444648 17070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:35.450850 17070 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:35.454349 17106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:35.455929 17070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:35.456123 17070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
uuid: "49ac0879298948648d31096ff564f34e"
format_stamp: "Formatted at 2026-08-11 02:57:25 on dist-test-slave-p3t5"
I20260811 02:57:35.457166 17070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:35.502672 17070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:35.503700 17070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:35.504046 17070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:35.513505 17122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:35.514001 17122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:35.549149 17122 leader_election.cc:288] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72eb236aa6043eb91a5dfce161d5057 (127.13.94.189:45939), e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188:44361), 49ac0879298948648d31096ff564f34e (127.13.94.187:44947), 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186:40871)
W20260811 02:57:35.555929 16819 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.94.187:44947: connect: Connection refused (error 111)
W20260811 02:57:35.566147 16817 leader_election.cc:334] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186:40871): Network error: Client connection negotiation failed: client connection to 127.13.94.186:40871: connect: Connection refused (error 111)
W20260811 02:57:35.566555 16819 leader_election.cc:334] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 49ac0879298948648d31096ff564f34e (127.13.94.187:44947): Network error: Client connection negotiation failed: client connection to 127.13.94.187:44947: connect: Connection refused (error 111)
I20260811 02:57:35.566385 16940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 4 candidate_status { last_received { term: 3 index: 66 } } ignore_live_leader: false dest_uuid: "d72eb236aa6043eb91a5dfce161d5057" is_pre_election: true
I20260811 02:57:35.567287 16940 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 3.
I20260811 02:57:35.570554 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 4 candidate_status { last_received { term: 3 index: 66 } } ignore_live_leader: false dest_uuid: "e3aa822be3af4af4a40b453c83d7d425" is_pre_election: true
I20260811 02:57:35.571317 17038 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 3.
I20260811 02:57:35.572034 16818 leader_election.cc:302] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 5 voters: 3 yes votes; 2 no votes. yes voters: 92aef555682a43a1a1619b2bc9ba7ce1, d72eb236aa6043eb91a5dfce161d5057, e3aa822be3af4af4a40b453c83d7d425; no voters: 213a74fcc5cd4d3f9be2f848f3b3fd49, 49ac0879298948648d31096ff564f34e
I20260811 02:57:35.572832 17122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 02:57:35.573005 17122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:35.573092 17122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:57:35.578924 17122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:35.580623 16940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 4 candidate_status { last_received { term: 3 index: 66 } } ignore_live_leader: false dest_uuid: "d72eb236aa6043eb91a5dfce161d5057"
I20260811 02:57:35.580928 16940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:57:35.581861 17122 leader_election.cc:288] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 4 election: Requested vote from peers d72eb236aa6043eb91a5dfce161d5057 (127.13.94.189:45939), e3aa822be3af4af4a40b453c83d7d425 (127.13.94.188:44361), 49ac0879298948648d31096ff564f34e (127.13.94.187:44947), 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186:40871)
W20260811 02:57:35.583763 16819 leader_election.cc:334] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 49ac0879298948648d31096ff564f34e (127.13.94.187:44947): Network error: Client connection negotiation failed: client connection to 127.13.94.187:44947: connect: Connection refused (error 111)
W20260811 02:57:35.584025 16817 leader_election.cc:334] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186:40871): Network error: Client connection negotiation failed: client connection to 127.13.94.186:40871: connect: Connection refused (error 111)
I20260811 02:57:35.584722 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" candidate_term: 4 candidate_status { last_received { term: 3 index: 66 } } ignore_live_leader: false dest_uuid: "e3aa822be3af4af4a40b453c83d7d425"
I20260811 02:57:35.585021 17038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:57:35.588378 16940 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 4.
I20260811 02:57:35.589638 17070 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.187:44947
I20260811 02:57:35.589600 17170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.187:44947 every 8 connection(s)
I20260811 02:57:35.591673 17070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-3/data/info.pb
I20260811 02:57:35.592482 17038 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92aef555682a43a1a1619b2bc9ba7ce1 in term 4.
I20260811 02:57:35.593132 16818 leader_election.cc:302] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 5 voters: 3 yes votes; 2 no votes. yes voters: 92aef555682a43a1a1619b2bc9ba7ce1, d72eb236aa6043eb91a5dfce161d5057, e3aa822be3af4af4a40b453c83d7d425; no voters: 213a74fcc5cd4d3f9be2f848f3b3fd49, 49ac0879298948648d31096ff564f34e
I20260811 02:57:35.593784 17122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 02:57:35.595105 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 17070
I20260811 02:57:35.597898 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.186:40871
--webserver_interface=127.13.94.186
--webserver_port=34273
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871 with env {}
I20260811 02:57:35.601824 17122 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [term 4 LEADER]: Becoming Leader. State: Replica: 92aef555682a43a1a1619b2bc9ba7ce1, State: Running, Role: LEADER
I20260811 02:57:35.602047 17171 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:35.602520 17122 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 66, Committed index: 66, Last appended: 3.66, Last appended by leader: 66, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:35.607895 17171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Bootstrap starting.
I20260811 02:57:35.608028 17173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:35.608503 17173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:35.610754 17176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:35.613837 17176 catalog_manager.cc:669] Loaded metadata for table default.1_4 [id=1765740dfbf5417e90b87056985f59e1]
I20260811 02:57:35.614390 17176 catalog_manager.cc:669] Loaded metadata for table default.1_8 [id=1b031c8533b1417591024c3b6a9b2b87]
I20260811 02:57:35.614758 17176 catalog_manager.cc:669] Loaded metadata for table default.0_4 [id=1dd6d5be9a9f4a6ba48fee38b5e87d63]
I20260811 02:57:35.615104 17176 catalog_manager.cc:669] Loaded metadata for table default.1_1 [id=2b6d1a80a21d49ec8120aeac1c9b2764]
I20260811 02:57:35.615451 17176 catalog_manager.cc:669] Loaded metadata for table default.0_8 [id=5ac90fca3b9d4650b0f568eef6113738]
I20260811 02:57:35.615775 17176 catalog_manager.cc:669] Loaded metadata for table default.0_2 [id=5bf294e9599741cab356594a0889edc3]
I20260811 02:57:35.616092 17176 catalog_manager.cc:669] Loaded metadata for table default.0_1 [id=6c64c679309b4f29ab1ba582a3bcff07]
I20260811 02:57:35.616418 17176 catalog_manager.cc:669] Loaded metadata for table default.0_5 [id=a539b8205c734c08b5cf1e5e1250b1b6]
I20260811 02:57:35.616803 17176 catalog_manager.cc:669] Loaded metadata for table default.1_2 [id=acb767fa16b7453da75b503e3c4be4f5]
I20260811 02:57:35.617141 17176 catalog_manager.cc:669] Loaded metadata for table default.1_5 [id=aea54d2174504fa89b4df2ff08a53688]
I20260811 02:57:35.617466 17176 catalog_manager.cc:669] Loaded metadata for table default.0_7 [id=ce943c1496344fab928bc054e7206385]
I20260811 02:57:35.617933 17176 catalog_manager.cc:669] Loaded metadata for table default.1_7 [id=d1f9862292724158acf991e7f1aa1f80]
I20260811 02:57:35.618333 17176 catalog_manager.cc:669] Loaded metadata for table test-workload [id=f0728c3f7ecc49b5ae49a5bd89ce2df8]
I20260811 02:57:35.622123 17176 tablet_loader.cc:96] loaded metadata for tablet 0c91c12fe10e45f8a8dacdadf9a332dc (table default.1_5 [id=aea54d2174504fa89b4df2ff08a53688])
I20260811 02:57:35.622550 17176 tablet_loader.cc:96] loaded metadata for tablet 11ebf0753b444c0fb704488c06204383 (table default.0_2 [id=5bf294e9599741cab356594a0889edc3])
I20260811 02:57:35.622932 17176 tablet_loader.cc:96] loaded metadata for tablet 13b5598d42214d908fe5b77e009b8939 (table default.0_8 [id=5ac90fca3b9d4650b0f568eef6113738])
I20260811 02:57:35.623319 17176 tablet_loader.cc:96] loaded metadata for tablet 3a186dc26d0246568bcd1be33c1aa781 (table default.0_5 [id=a539b8205c734c08b5cf1e5e1250b1b6])
I20260811 02:57:35.623664 17176 tablet_loader.cc:96] loaded metadata for tablet 420ed2b9216647498d9a2c956475ce4c (table default.0_4 [id=1dd6d5be9a9f4a6ba48fee38b5e87d63])
I20260811 02:57:35.623976 17176 tablet_loader.cc:96] loaded metadata for tablet 4694adb271df417297bb20aed3fbe6ac (table default.0_7 [id=ce943c1496344fab928bc054e7206385])
I20260811 02:57:35.624316 17176 tablet_loader.cc:96] loaded metadata for tablet a4ff2780d1b54a829f5a65f9b36d10ec (table default.0_1 [id=6c64c679309b4f29ab1ba582a3bcff07])
I20260811 02:57:35.624650 17176 tablet_loader.cc:96] loaded metadata for tablet a6142c6b82194a798989a9d808341f60 (table default.1_8 [id=1b031c8533b1417591024c3b6a9b2b87])
I20260811 02:57:35.624979 17176 tablet_loader.cc:96] loaded metadata for tablet a65dd69fcfaa429697f56a4329c97145 (table default.1_7 [id=d1f9862292724158acf991e7f1aa1f80])
I20260811 02:57:35.625314 17176 tablet_loader.cc:96] loaded metadata for tablet e55d01af74a84dc391101460f2a9c063 (table default.1_2 [id=acb767fa16b7453da75b503e3c4be4f5])
I20260811 02:57:35.625720 17176 tablet_loader.cc:96] loaded metadata for tablet ee6c637392954d28abe1172c900498b2 (table default.1_1 [id=2b6d1a80a21d49ec8120aeac1c9b2764])
I20260811 02:57:35.626050 17176 tablet_loader.cc:96] loaded metadata for tablet f2e8bb100c3f41d5b74490a795fe638e (table default.1_4 [id=1765740dfbf5417e90b87056985f59e1])
I20260811 02:57:35.626379 17176 tablet_loader.cc:96] loaded metadata for tablet f78e24d030f549fb86183de598e3576b (table test-workload [id=f0728c3f7ecc49b5ae49a5bd89ce2df8])
I20260811 02:57:35.626724 17176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:35.628628 17176 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:35.628734 17176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:35.630518 17176 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:35.632829 17176 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1: Loaded TSK: 0
I20260811 02:57:35.634182 17176 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:35.655582 17171 log.cc:824] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:35.696660 16835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51508:
name: "default.0_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:35.704373 16835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51520:
name: "default.1_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:35.723830 17171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Bootstrap replayed 1/1 log segments. Stats: ops{read=66 overwritten=0 applied=66 ignored=0} inserts{seen=29 ignored=0} mutations{seen=45 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:35.724606 17171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: Bootstrap complete.
I20260811 02:57:35.739077 17171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:35.740505 17171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49ac0879298948648d31096ff564f34e, State: Initialized, Role: FOLLOWER
I20260811 02:57:35.741398 17171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 66, Last appended: 3.66, Last appended by leader: 66, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:35.742476 17179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:35.743026 17179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:35.743824 17171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:35.768833 17190 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:35.769040 17190 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: loading cluster ID for follower catalog manager: success
I20260811 02:57:35.773954 17190 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: acquiring CA information for follower catalog manager: success
I20260811 02:57:35.776985 17190 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:35.816610 17070 master_runner.cc:186] Error getting master registration for 127.13.94.186:40871: Network error: Client connection negotiation failed: client connection to 127.13.94.186:40871: connect: Connection refused (error 111),
I20260811 02:57:35.823163 17070 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:57:36.043818 16940 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [term 4 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 3 index: 66. Preceding OpId from leader: term: 4 index: 67. (index mismatch)
I20260811 02:57:36.045162 17173 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20260811 02:57:36.059293 17196 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.059839 17196 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 02:57:36.078567 17175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:36.078953 17175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:36.079116 17175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 02:57:36.079236 17038 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [term 4 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 3 index: 66. Preceding OpId from leader: term: 4 index: 67. (index mismatch)
I20260811 02:57:36.080188 17173 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20260811 02:57:36.084586 17201 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.085043 17201 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 02:57:36.091400 17175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:36.091562 17175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:36.091648 17175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:36.091746 17175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:36.099732 17201 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.100124 17201 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aa822be3af4af4a40b453c83d7d425 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 02:57:36.110620 16817 consensus_peers.cc:487] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 -> Peer 213a74fcc5cd4d3f9be2f848f3b3fd49 (127.13.94.186:40871): Couldn't send request to peer 213a74fcc5cd4d3f9be2f848f3b3fd49. Status: Network error: Client connection negotiation failed: client connection to 127.13.94.186:40871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 02:57:36.110905 17175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.186:40871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/info.pb
--webserver_interface=127.13.94.186
--webserver_port=34273
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:36.112659 17175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:36.113216 17173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.113566 17173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:36.113124 17122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.114146 17122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:36.115345 17175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:36.121570 17196 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.121968 17196 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72eb236aa6043eb91a5dfce161d5057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:36.124307 17139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:57:36.129145 17139 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [term 4 FOLLOWER]: Refusing update from remote peer 92aef555682a43a1a1619b2bc9ba7ce1: Log matching property violated. Preceding OpId in replica: term: 3 index: 66. Preceding OpId from leader: term: 4 index: 67. (index mismatch)
I20260811 02:57:36.130245 17206 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
W20260811 02:57:36.134126 17214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:36.134574 17215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:36.136363 17219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:36.140033 17175 server_base.cc:1034] running on GCE node
I20260811 02:57:36.141175 17175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:36.141139 17179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.141508 17179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:36.141044 17221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.141889 17221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ac0879298948648d31096ff564f34e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:36.143069 17175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:36.144372 17175 hybrid_clock.cc:648] HybridClock initialized: now 1786417056144280 us; error 64 us; skew 500 ppm
I20260811 02:57:36.144802 17175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:36.147796 17175 webserver.cc:460] Webserver started at http://127.13.94.186:34273/ using document root <none> and password file <none>
I20260811 02:57:36.148571 17175 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:36.148736 17175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:36.155196 17175 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:36.158725 17227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:36.160537 17175 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:36.160781 17175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-p3t5"
I20260811 02:57:36.162015 17175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:36.185225 17175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:36.186314 17175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:36.186678 17175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:36.209045 17175 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.186:40871
I20260811 02:57:36.209070 17281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.186:40871 every 8 connection(s)
I20260811 02:57:36.210843 17175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/master-4/data/info.pb
I20260811 02:57:36.216589 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 17175
I20260811 02:57:36.217275 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:37837
--local_ip_for_outbound_sockets=127.13.94.129
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939
--webserver_port=34685
--webserver_interface=127.13.94.129
--builtin_ntp_servers=127.13.94.148:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871 with env {}
I20260811 02:57:36.221113 17282 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:36.226806 17282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Bootstrap starting.
I20260811 02:57:36.260465 17282 log.cc:824] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:36.311806 17282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Bootstrap replayed 1/1 log segments. Stats: ops{read=66 overwritten=0 applied=66 ignored=0} inserts{seen=29 ignored=0} mutations{seen=45 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:36.312465 17282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: Bootstrap complete.
I20260811 02:57:36.322494 17282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:36.323640 17282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213a74fcc5cd4d3f9be2f848f3b3fd49, State: Initialized, Role: FOLLOWER
I20260811 02:57:36.324450 17282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 66, Last appended: 3.66, Last appended by leader: 66, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } }
I20260811 02:57:36.325507 17287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.326097 17287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:36.326867 17282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:36.352068 17298 catalog_manager.cc:1259] Loaded cluster ID: acb9258085d447cd943dcabafea772e1
I20260811 02:57:36.352208 17298 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: loading cluster ID for follower catalog manager: success
I20260811 02:57:36.356197 17298 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: acquiring CA information for follower catalog manager: success
I20260811 02:57:36.359404 17298 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:36.499977 17257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:57:36.511544 17307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.512351 17307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:36.512359 17287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aef555682a43a1a1619b2bc9ba7ce1. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.512835 17287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213a74fcc5cd4d3f9be2f848f3b3fd49 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:36.532567 17206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:36.533028 17206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
W20260811 02:57:36.630867 17285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:36.631172 17285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:36.631314 17285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:36.639211 17285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:36.639457 17285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:57:36.650254 17285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:37837
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=34685
--tserver_master_addrs=127.13.94.190:38889,127.13.94.189:45939,127.13.94.188:44361,127.13.94.187:44947,127.13.94.186:40871
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:36.652276 17285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:36.654381 17285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:36.671615 17315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:36.671711 17313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:36.672515 17312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:36.672928 17285 server_base.cc:1034] running on GCE node
I20260811 02:57:36.673751 17285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:36.674918 17285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:36.676182 17285 hybrid_clock.cc:648] HybridClock initialized: now 1786417056676086 us; error 67 us; skew 500 ppm
I20260811 02:57:36.676537 17285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:36.679409 17285 webserver.cc:460] Webserver started at http://127.13.94.129:34685/ using document root <none> and password file <none>
I20260811 02:57:36.680215 17285 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:36.680369 17285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:36.686949 17285 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:57:36.691769 17322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:36.693511 17285 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 02:57:36.693742 17285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "42a22db0f41a4489a0dc8ce5315e4428"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-p3t5"
I20260811 02:57:36.694756 17285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:36.730036 17285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:36.731426 17285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:36.731774 17285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:36.732129 16835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40610:
name: "default.0_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:36.746193 17285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:36.764513 17343 ts_tablet_manager.cc:542] Loading tablet metadata (0/13 complete)
I20260811 02:57:36.777828 16835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40612:
name: "default.1_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:36.810743 17285 ts_tablet_manager.cc:579] Loaded tablet metadata (13 total tablets, 13 live tablets)
I20260811 02:57:36.810892 17285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.049s user 0.000s sys 0.004s
I20260811 02:57:36.811187 17285 ts_tablet_manager.cc:594] Registering tablets (0/13 complete)
I20260811 02:57:36.815253 17343 tablet_bootstrap.cc:492] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:36.833431 17343 log.cc:824] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:36.837945 17343 tablet_bootstrap.cc:492] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:36.838670 17343 tablet_bootstrap.cc:492] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:36.839385 17285 ts_tablet_manager.cc:610] Registered 13 tablets
I20260811 02:57:36.839494 17285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.028s user 0.025s sys 0.003s
I20260811 02:57:36.840006 17343 ts_tablet_manager.cc:1397] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.012s
I20260811 02:57:36.852636 17343 raft_consensus.cc:348] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.853607 17343 raft_consensus.cc:729] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:36.854607 17343 consensus_queue.cc:260] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.854861 17343 raft_consensus.cc:388] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:36.855042 17343 raft_consensus.cc:482] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:36.855248 17343 raft_consensus.cc:3037] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:36.859539 17343 raft_consensus.cc:504] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.860157 17343 leader_election.cc:302] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:36.861958 17343 leader_election.cc:288] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:57:36.862080 17375 raft_consensus.cc:2781] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:36.863497 17375 raft_consensus.cc:686] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:36.864086 17343 ts_tablet_manager.cc:1428] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.024s user 0.013s sys 0.012s
I20260811 02:57:36.864132 17375 consensus_queue.cc:237] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.864569 17343 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:36.883659 17343 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:36.884528 17343 tablet_bootstrap.cc:492] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:36.886255 17343 ts_tablet_manager.cc:1397] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.011s
I20260811 02:57:36.887858 17343 raft_consensus.cc:348] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.888190 17343 raft_consensus.cc:729] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:36.888762 17343 consensus_queue.cc:260] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.889055 17343 raft_consensus.cc:388] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:36.889257 17343 raft_consensus.cc:482] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:36.889420 17343 raft_consensus.cc:3037] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:57:36.893500 17343 raft_consensus.cc:504] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.893923 17343 leader_election.cc:302] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:36.894301 17375 raft_consensus.cc:2781] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 02:57:36.894539 17375 raft_consensus.cc:686] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:36.894865 17375 consensus_queue.cc:237] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.895639 17343 leader_election.cc:288] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 4 election: Requested vote from peers
I20260811 02:57:36.901988 17343 ts_tablet_manager.cc:1428] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.016s user 0.008s sys 0.001s
I20260811 02:57:36.902320 17343 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:36.913023 17285 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:37837
I20260811 02:57:36.913095 17470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:37837 every 8 connection(s)
I20260811 02:57:36.915113 17285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:36.918603 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 17285
I20260811 02:57:36.924981 17343 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:36.925606 17343 tablet_bootstrap.cc:492] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:36.927174 17343 ts_tablet_manager.cc:1397] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.025s user 0.016s sys 0.007s
I20260811 02:57:36.928279 17343 raft_consensus.cc:348] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.928542 17343 raft_consensus.cc:729] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:36.928841 17343 consensus_queue.cc:260] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.929122 17343 raft_consensus.cc:388] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:36.929239 17343 raft_consensus.cc:482] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:36.929405 17343 raft_consensus.cc:3037] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:57:36.933230 17343 raft_consensus.cc:504] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.933560 17343 leader_election.cc:302] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:36.933923 17381 raft_consensus.cc:2781] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 02:57:36.934154 17381 raft_consensus.cc:686] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:36.934520 17381 consensus_queue.cc:237] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.938120 17343 leader_election.cc:288] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 4 election: Requested vote from peers
I20260811 02:57:36.939002 17343 ts_tablet_manager.cc:1428] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.012s user 0.009s sys 0.000s
I20260811 02:57:36.939474 17343 tablet_bootstrap.cc:492] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:36.960228 17343 tablet_bootstrap.cc:492] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:36.960772 17343 tablet_bootstrap.cc:492] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:36.961977 17343 ts_tablet_manager.cc:1397] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.011s
I20260811 02:57:36.962715 17343 raft_consensus.cc:348] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.962903 17343 raft_consensus.cc:729] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:36.963210 17343 consensus_queue.cc:260] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.963440 17343 raft_consensus.cc:388] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:36.963569 17343 raft_consensus.cc:482] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:36.963683 17343 raft_consensus.cc:3037] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:36.966696 17343 raft_consensus.cc:504] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.967056 17343 leader_election.cc:302] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:36.967332 17381 raft_consensus.cc:2781] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:36.967537 17381 raft_consensus.cc:686] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:36.967833 17381 consensus_queue.cc:237] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:36.973466 17343 leader_election.cc:288] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:36.974552 17343 ts_tablet_manager.cc:1428] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260811 02:57:36.974926 17343 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:36.985288 17471 heartbeater.cc:347] Connected to a master server at 127.13.94.186:40871
I20260811 02:57:36.985881 17471 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:36.987011 17471 heartbeater.cc:511] Master 127.13.94.186:40871 requested a full tablet report, sending...
I20260811 02:57:36.993989 17473 heartbeater.cc:347] Connected to a master server at 127.13.94.187:44947
I20260811 02:57:36.994187 17473 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:36.994758 17473 heartbeater.cc:511] Master 127.13.94.187:44947 requested a full tablet report, sending...
I20260811 02:57:37.005067 17127 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:37.005038 17245 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:37.011879 17484 heartbeater.cc:347] Connected to a master server at 127.13.94.190:38889
I20260811 02:57:37.012082 17484 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:37.012669 17484 heartbeater.cc:511] Master 127.13.94.190:38889 requested a full tablet report, sending...
I20260811 02:57:37.016554 16835 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:37.020136 16835 catalog_manager.cc:5495] T f78e24d030f549fb86183de598e3576b P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.021405 16835 catalog_manager.cc:5495] T e55d01af74a84dc391101460f2a9c063 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.021950 16835 catalog_manager.cc:5495] T a6142c6b82194a798989a9d808341f60 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.022406 16835 catalog_manager.cc:5495] T 420ed2b9216647498d9a2c956475ce4c P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.041225 17478 heartbeater.cc:347] Connected to a master server at 127.13.94.188:44361
I20260811 02:57:37.041426 17478 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:37.042093 17478 heartbeater.cc:511] Master 127.13.94.188:44361 requested a full tablet report, sending...
I20260811 02:57:37.045689 17122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "92aef555682a43a1a1619b2bc9ba7ce1" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 38889 } } peers { permanent_uuid: "d72eb236aa6043eb91a5dfce161d5057" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 45939 } } peers { permanent_uuid: "e3aa822be3af4af4a40b453c83d7d425" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 44361 } attrs { promote: false } } peers { permanent_uuid: "49ac0879298948648d31096ff564f34e" member_type: VOTER last_known_addr { host: "127.13.94.187" port: 44947 } attrs { promote: false } } peers { permanent_uuid: "213a74fcc5cd4d3f9be2f848f3b3fd49" member_type: VOTER last_known_addr { host: "127.13.94.186" port: 40871 } attrs { promote: false } } }
I20260811 02:57:37.046267 17122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aef555682a43a1a1619b2bc9ba7ce1 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:37.046947 17025 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:37.066504 17486 heartbeater.cc:347] Connected to a master server at 127.13.94.189:45939
I20260811 02:57:37.066782 17486 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:37.067435 17486 heartbeater.cc:511] Master 127.13.94.189:45939 requested a full tablet report, sending...
I20260811 02:57:37.072644 16929 ts_manager.cc:194] Registered new tserver with Master: 42a22db0f41a4489a0dc8ce5315e4428 (127.13.94.129:37837)
I20260811 02:57:37.078100 17343 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:37.078704 17343 tablet_bootstrap.cc:492] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:37.080027 17343 ts_tablet_manager.cc:1397] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.105s user 0.045s sys 0.037s
I20260811 02:57:37.081079 17343 raft_consensus.cc:348] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.081318 17343 raft_consensus.cc:729] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:37.081679 17343 consensus_queue.cc:260] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.081936 17343 raft_consensus.cc:388] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:37.082056 17343 raft_consensus.cc:482] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:37.082156 17343 raft_consensus.cc:3037] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:57:37.085212 17343 raft_consensus.cc:504] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.085575 17343 leader_election.cc:302] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:37.085963 17375 raft_consensus.cc:2781] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 02:57:37.086128 17375 raft_consensus.cc:686] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:37.086431 17375 consensus_queue.cc:237] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.091116 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:37.092726 17343 leader_election.cc:288] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 4 election: Requested vote from peers
I20260811 02:57:37.093931 17343 ts_tablet_manager.cc:1428] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.014s user 0.005s sys 0.004s
I20260811 02:57:37.094298 17343 tablet_bootstrap.cc:492] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:37.107892 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:37.140347 17343 tablet_bootstrap.cc:492] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:37.140870 17343 tablet_bootstrap.cc:492] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:37.142102 17343 ts_tablet_manager.cc:1397] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.048s user 0.010s sys 0.020s
I20260811 02:57:37.142992 17343 raft_consensus.cc:348] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.143224 17343 raft_consensus.cc:729] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:37.143528 17343 consensus_queue.cc:260] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.143743 17343 raft_consensus.cc:388] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:37.143868 17343 raft_consensus.cc:482] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:37.143951 17343 raft_consensus.cc:3037] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:37.146000 16835 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.94.129:39215
I20260811 02:57:37.146544 17343 raft_consensus.cc:504] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.146878 17343 leader_election.cc:302] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:37.147121 17343 leader_election.cc:288] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:37.147184 17375 raft_consensus.cc:2781] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:37.147432 17375 raft_consensus.cc:686] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:37.147671 17375 consensus_queue.cc:237] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.147969 17343 ts_tablet_manager.cc:1428] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 02:57:37.148275 17343 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:37.150259 17484 heartbeater.cc:503] Master 127.13.94.190:38889 was elected leader, sending a full tablet report...
I20260811 02:57:37.157442 16835 catalog_manager.cc:5495] T a65dd69fcfaa429697f56a4329c97145 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.158031 16835 catalog_manager.cc:5495] T 11ebf0753b444c0fb704488c06204383 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.170603 17343 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:37.171103 17343 tablet_bootstrap.cc:492] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:37.172065 17343 ts_tablet_manager.cc:1397] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.008s
I20260811 02:57:37.172731 17343 raft_consensus.cc:348] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.172911 17343 raft_consensus.cc:729] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:37.173102 17343 consensus_queue.cc:260] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.173240 17343 raft_consensus.cc:388] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:37.173310 17343 raft_consensus.cc:482] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:37.173378 17343 raft_consensus.cc:3037] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:57:37.175315 17343 raft_consensus.cc:504] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.175585 17343 leader_election.cc:302] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:37.175812 17343 leader_election.cc:288] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 4 election: Requested vote from peers
I20260811 02:57:37.175864 17375 raft_consensus.cc:2781] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 02:57:37.176072 17375 raft_consensus.cc:686] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:37.176306 17343 ts_tablet_manager.cc:1428] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:37.176343 17375 consensus_queue.cc:237] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.176620 17343 tablet_bootstrap.cc:492] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:37.184021 16835 catalog_manager.cc:5495] T a4ff2780d1b54a829f5a65f9b36d10ec P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.196511 17343 tablet_bootstrap.cc:492] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:37.196942 17343 tablet_bootstrap.cc:492] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:37.198091 17343 ts_tablet_manager.cc:1397] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.007s
I20260811 02:57:37.198913 17343 raft_consensus.cc:348] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.199110 17343 raft_consensus.cc:729] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:37.199378 17343 consensus_queue.cc:260] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.199563 17343 raft_consensus.cc:388] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:37.199688 17343 raft_consensus.cc:482] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:37.199808 17343 raft_consensus.cc:3037] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:37.202189 17343 raft_consensus.cc:504] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.202498 17343 leader_election.cc:302] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:37.202721 17343 leader_election.cc:288] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:57:37.202759 17375 raft_consensus.cc:2781] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:37.203246 17343 ts_tablet_manager.cc:1428] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260811 02:57:37.203002 17375 raft_consensus.cc:686] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:37.204048 17343 tablet_bootstrap.cc:492] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:37.204017 17375 consensus_queue.cc:237] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.209034 16835 catalog_manager.cc:5495] T 3a186dc26d0246568bcd1be33c1aa781 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.231986 17343 tablet_bootstrap.cc:492] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:37.232506 17343 tablet_bootstrap.cc:492] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:37.233536 17343 ts_tablet_manager.cc:1397] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.030s user 0.011s sys 0.015s
I20260811 02:57:37.234239 17343 raft_consensus.cc:348] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.234409 17343 raft_consensus.cc:729] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:37.234720 17343 consensus_queue.cc:260] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.234953 17343 raft_consensus.cc:388] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:37.235092 17343 raft_consensus.cc:482] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:37.235224 17343 raft_consensus.cc:3037] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:37.237308 17343 raft_consensus.cc:504] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.237582 17343 leader_election.cc:302] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:37.237850 17343 leader_election.cc:288] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:57:37.237960 17375 raft_consensus.cc:2781] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:37.238157 17375 raft_consensus.cc:686] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:37.238384 17343 ts_tablet_manager.cc:1428] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:57:37.238461 17375 consensus_queue.cc:237] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.238653 17343 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:37.249763 16835 catalog_manager.cc:5495] T f2e8bb100c3f41d5b74490a795fe638e P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.262265 17343 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:37.262864 17343 tablet_bootstrap.cc:492] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:37.264153 17343 ts_tablet_manager.cc:1397] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.017s
I20260811 02:57:37.265039 17343 raft_consensus.cc:348] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.265290 17343 raft_consensus.cc:729] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:37.265559 17343 consensus_queue.cc:260] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.265802 17343 raft_consensus.cc:388] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:37.265905 17343 raft_consensus.cc:482] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:37.266057 17343 raft_consensus.cc:3037] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:57:37.267980 17343 raft_consensus.cc:504] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.268218 17343 leader_election.cc:302] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:37.268455 17343 leader_election.cc:288] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 4 election: Requested vote from peers
I20260811 02:57:37.268503 17375 raft_consensus.cc:2781] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 02:57:37.268729 17375 raft_consensus.cc:686] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [term 4 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:37.269030 17343 ts_tablet_manager.cc:1428] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260811 02:57:37.268957 17375 consensus_queue.cc:237] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.269297 17343 tablet_bootstrap.cc:492] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:37.271806 16835 catalog_manager.cc:5495] T ee6c637392954d28abe1172c900498b2 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.286558 17343 tablet_bootstrap.cc:492] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:37.287228 17343 tablet_bootstrap.cc:492] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:37.288304 17343 ts_tablet_manager.cc:1397] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.007s
I20260811 02:57:37.288997 17343 raft_consensus.cc:348] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.289196 17343 raft_consensus.cc:729] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:37.289498 17343 consensus_queue.cc:260] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.289719 17343 raft_consensus.cc:388] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:37.289794 17343 raft_consensus.cc:482] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:37.289855 17343 raft_consensus.cc:3037] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:37.291751 17343 raft_consensus.cc:504] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.291956 17343 leader_election.cc:302] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:37.292155 17343 leader_election.cc:288] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:37.292212 17375 raft_consensus.cc:2781] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:37.292343 17375 raft_consensus.cc:686] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:37.292591 17375 consensus_queue.cc:237] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.292706 17343 ts_tablet_manager.cc:1428] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:37.292986 17343 tablet_bootstrap.cc:492] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:37.302137 16835 catalog_manager.cc:5495] T 4694adb271df417297bb20aed3fbe6ac P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.315017 17343 tablet_bootstrap.cc:492] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:37.315534 17343 tablet_bootstrap.cc:492] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:37.316552 17343 ts_tablet_manager.cc:1397] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.009s
I20260811 02:57:37.317258 17343 raft_consensus.cc:348] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.317474 17343 raft_consensus.cc:729] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:37.317819 17343 consensus_queue.cc:260] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.318010 17343 raft_consensus.cc:388] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:37.318132 17343 raft_consensus.cc:482] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:37.318257 17343 raft_consensus.cc:3037] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:37.320188 17343 raft_consensus.cc:504] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.320461 17343 leader_election.cc:302] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:37.320683 17343 leader_election.cc:288] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:57:37.320765 17375 raft_consensus.cc:2781] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:37.321044 17375 raft_consensus.cc:686] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [term 3 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:37.321190 17343 ts_tablet_manager.cc:1428] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:37.321283 17375 consensus_queue.cc:237] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.321463 17343 tablet_bootstrap.cc:492] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap starting.
I20260811 02:57:37.324890 16835 catalog_manager.cc:5495] T 0c91c12fe10e45f8a8dacdadf9a332dc P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.340526 17343 tablet_bootstrap.cc:492] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:37.341046 17343 tablet_bootstrap.cc:492] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428: Bootstrap complete.
I20260811 02:57:37.342144 17343 ts_tablet_manager.cc:1397] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.013s
I20260811 02:57:37.342841 17343 raft_consensus.cc:348] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.343055 17343 raft_consensus.cc:729] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Initialized, Role: FOLLOWER
I20260811 02:57:37.343331 17343 consensus_queue.cc:260] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.343493 17343 raft_consensus.cc:388] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:37.343568 17343 raft_consensus.cc:482] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:37.343636 17343 raft_consensus.cc:3037] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:37.345523 17343 raft_consensus.cc:504] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.345777 17343 leader_election.cc:302] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a22db0f41a4489a0dc8ce5315e4428; no voters:
I20260811 02:57:37.345975 17343 leader_election.cc:288] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:37.346045 17375 raft_consensus.cc:2781] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:37.346192 17375 raft_consensus.cc:686] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [term 2 LEADER]: Becoming Leader. State: Replica: 42a22db0f41a4489a0dc8ce5315e4428, State: Running, Role: LEADER
I20260811 02:57:37.346575 17343 ts_tablet_manager.cc:1428] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260811 02:57:37.346452 17375 consensus_queue.cc:237] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } }
I20260811 02:57:37.351647 16835 catalog_manager.cc:5495] T 13b5598d42214d908fe5b77e009b8939 P 42a22db0f41a4489a0dc8ce5315e4428 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "42a22db0f41a4489a0dc8ce5315e4428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a22db0f41a4489a0dc8ce5315e4428" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 37837 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:37.780995 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 17285
I20260811 02:57:37.791204 17466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:37.905786 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 17285
I20260811 02:57:37.938529 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 16784
I20260811 02:57:37.939890 16870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:38.057883 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 16784
W20260811 02:57:38.085475 16915 connection.cc:535] server connection from 127.0.0.1:37854 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 02:57:38.086068 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 16877
I20260811 02:57:38.087401 16961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:38.204648 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 16877
I20260811 02:57:38.238677 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 16969
I20260811 02:57:38.240005 17060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:38.354434 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 16969
I20260811 02:57:38.379980 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 17070
I20260811 02:57:38.381224 17166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:38.491004 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 17070
I20260811 02:57:38.515264 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 17175
I20260811 02:57:38.516499 17277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:38.630151 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 17175
2026-08-11T02:57:38Z chronyd exiting
[ OK ] AutoAddMasterTest.TestAddWithOnGoingDdl (20362 ms)
[----------] 1 test from AutoAddMasterTest (20362 ms total)
[----------] 1 test from ParameterizedAddMasterTest
[ RUN ] ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL/3
Loading random data
Initializing database '/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:38 dist-test-slave-p3t5 krb5kdc[17539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:38 dist-test-slave-p3t5 krb5kdc[17539](info): set up 2 sockets
Aug 11 02:57:38 dist-test-slave-p3t5 krb5kdc[17539](info): commencing operation
krb5kdc: starting...
W20260811 02:57:39.855021 13690 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.005s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/test-user.keytab.
Aug 11 02:57:40 dist-test-slave-p3t5 krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.94.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.94.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab.
Entry for principal kudu/127.13.94.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab.
I20260811 02:57:40.114444 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:41293
--webserver_interface=127.13.94.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab
--principal=kudu/127.13.94.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.94.148:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:41293,127.13.94.189:40361
--master_auto_join_cluster=false with env {KRB5CCNAME=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:40.470005 17553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:40.470399 17553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:40.470557 17553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:40.478163 17553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:40.478351 17553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:40.478442 17553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:40.478551 17553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:40.489113 17553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:41293,127.13.94.189:40361
--master_auto_join_cluster=false
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab
--principal=kudu/127.13.94.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:41293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:57:40.489578 17553 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:41293,127.13.94.189:40361'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:57:40.490840 17553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:40.492355 17553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:40.505111 17560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:40.505038 17563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:40.505049 17561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:40.505800 17553 server_base.cc:1034] running on GCE node
I20260811 02:57:40.506731 17553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:40.508509 17553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:40.509724 17553 hybrid_clock.cc:648] HybridClock initialized: now 1786417060509661 us; error 66 us; skew 500 ppm
I20260811 02:57:40.510154 17553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 02:57:40 dist-test-slave-p3t5 krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:40.513803 17553 init.cc:377] Logged in from keytab as kudu/127.13.94.190@KRBTEST.COM (short username kudu)
I20260811 02:57:40.515471 17553 webserver.cc:460] Webserver started at http://127.13.94.190:40059/ using document root <none> and password file <none>
I20260811 02:57:40.516199 17553 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:40.516309 17553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:40.516580 17553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:40.519666 17553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/instance:
uuid: "17fb94eb8b7b4caea6f9669c114504f4"
format_stamp: "Formatted at 2026-08-11 02:57:40 on dist-test-slave-p3t5"
I20260811 02:57:40.520430 17553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal/instance:
uuid: "17fb94eb8b7b4caea6f9669c114504f4"
format_stamp: "Formatted at 2026-08-11 02:57:40 on dist-test-slave-p3t5"
I20260811 02:57:40.526245 17553 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 02:57:40.530177 17570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:40.531785 17553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:57:40.532011 17553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "17fb94eb8b7b4caea6f9669c114504f4"
format_stamp: "Formatted at 2026-08-11 02:57:40 on dist-test-slave-p3t5"
I20260811 02:57:40.532236 17553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:40.544724 17553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:40.550904 17553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:40.551249 17553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:40.575615 17553 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:41293
I20260811 02:57:40.575680 17626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:41293 every 8 connection(s)
I20260811 02:57:40.577185 17553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:40.580029 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 17553
I20260811 02:57:40.580272 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal/instance
I20260811 02:57:40.582226 17627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:40.591773 17627 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.94.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.94.189@KRBTEST.COM" created.
Aug 11 02:57:40 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.190@KRBTEST.COM for kudu/127.13.94.190@KRBTEST.COM
I20260811 02:57:40.619114 17627 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } has no permanent_uuid. Determining permanent_uuid...
W20260811 02:57:40.620270 17573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.94.189:40361: connect: Connection refused (error 111)
W20260811 02:57:40.622098 17627 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:40361: Network error: Client connection negotiation failed: client connection to 127.13.94.189:40361: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab.
Entry for principal kudu/127.13.94.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab.
I20260811 02:57:40.637481 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:40361
--webserver_interface=127.13.94.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab
--principal=kudu/127.13.94.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.94.148:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:41293,127.13.94.189:40361
--master_auto_join_cluster=false with env {KRB5CCNAME=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:57:40.671556 17627 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } attempt: 1
W20260811 02:57:40.674000 17627 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:40361: Network error: Client connection negotiation failed: client connection to 127.13.94.189:40361: connect: Connection refused (error 111)
I20260811 02:57:40.742337 17627 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } attempt: 2
W20260811 02:57:40.744704 17627 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:40361: Network error: Client connection negotiation failed: client connection to 127.13.94.189:40361: connect: Connection refused (error 111)
I20260811 02:57:40.836028 17627 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } attempt: 3
W20260811 02:57:40.838266 17627 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:40361: Network error: Client connection negotiation failed: client connection to 127.13.94.189:40361: connect: Connection refused (error 111)
I20260811 02:57:40.981609 17627 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } attempt: 4
W20260811 02:57:40.984030 17627 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:40361: Network error: Client connection negotiation failed: client connection to 127.13.94.189:40361: connect: Connection refused (error 111)
W20260811 02:57:40.992026 17634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:40.992360 17634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:40.992450 17634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:40.999918 17634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:41.000059 17634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:41.000105 17634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:41.000167 17634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:41.010875 17634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:41293,127.13.94.189:40361
--master_auto_join_cluster=false
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab
--principal=kudu/127.13.94.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:40361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:57:41.011235 17634 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:41293,127.13.94.189:40361'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:57:41.012352 17634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:41.013866 17634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:41.027072 17641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:41.027109 17645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:41.027132 17643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:41.027972 17634 server_base.cc:1034] running on GCE node
I20260811 02:57:41.028965 17634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:41.030681 17634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:41.031983 17634 hybrid_clock.cc:648] HybridClock initialized: now 1786417061031896 us; error 78 us; skew 500 ppm
I20260811 02:57:41.032444 17634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 02:57:41 dist-test-slave-p3t5 krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417061, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:41.036015 17634 init.cc:377] Logged in from keytab as kudu/127.13.94.189@KRBTEST.COM (short username kudu)
I20260811 02:57:41.037860 17634 webserver.cc:460] Webserver started at http://127.13.94.189:45441/ using document root <none> and password file <none>
I20260811 02:57:41.038635 17634 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:41.038794 17634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:41.039124 17634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:41.042135 17634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/instance:
uuid: "91d3a9f0f5474fb9b0409772125153e2"
format_stamp: "Formatted at 2026-08-11 02:57:41 on dist-test-slave-p3t5"
I20260811 02:57:41.042990 17634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal/instance:
uuid: "91d3a9f0f5474fb9b0409772125153e2"
format_stamp: "Formatted at 2026-08-11 02:57:41 on dist-test-slave-p3t5"
I20260811 02:57:41.048820 17634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 02:57:41.052626 17652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:41.054167 17634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:57:41.054368 17634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "91d3a9f0f5474fb9b0409772125153e2"
format_stamp: "Formatted at 2026-08-11 02:57:41 on dist-test-slave-p3t5"
I20260811 02:57:41.054601 17634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:41.068284 17634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:41.073194 17634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:41.073534 17634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:41.093541 17634 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:40361
I20260811 02:57:41.093585 17715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:40361 every 8 connection(s)
I20260811 02:57:41.095081 17634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:57:41.099090 17716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:41.102921 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 17634
I20260811 02:57:41.103129 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal/instance
I20260811 02:57:41.106885 17716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:57:41 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.94.190@KRBTEST.COM
Aug 11 02:57:41 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417061, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.189@KRBTEST.COM for kudu/127.13.94.190@KRBTEST.COM
I20260811 02:57:41.131065 17716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:57:41 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.94.189@KRBTEST.COM
Aug 11 02:57:41 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417061, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.189@KRBTEST.COM for kudu/127.13.94.189@KRBTEST.COM
I20260811 02:57:41.151942 17716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: Bootstrap starting.
I20260811 02:57:41.155524 17716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:41.156682 17716 log.cc:824] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:41.159641 17716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: No bootstrap required, opened a new log
I20260811 02:57:41.167434 17716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } }
I20260811 02:57:41.167824 17716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:41.167953 17716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91d3a9f0f5474fb9b0409772125153e2, State: Initialized, Role: FOLLOWER
I20260811 02:57:41.168712 17716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } }
I20260811 02:57:41.169595 17724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } }
I20260811 02:57:41.169991 17724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:41.170795 17716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:41.179550 17736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:41.179721 17736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:41.283375 17627 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } attempt: 5
Aug 11 02:57:41 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.190@KRBTEST.COM for kudu/127.13.94.189@KRBTEST.COM
I20260811 02:57:41.305145 17627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: Bootstrap starting.
I20260811 02:57:41.309684 17627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:41.311277 17627 log.cc:824] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:41.314713 17627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: No bootstrap required, opened a new log
W20260811 02:57:41.325896 17660 tablet.cc:2369] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:41.326256 17627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } }
I20260811 02:57:41.326715 17627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:41.326851 17627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fb94eb8b7b4caea6f9669c114504f4, State: Initialized, Role: FOLLOWER
I20260811 02:57:41.327600 17627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } }
I20260811 02:57:41.328517 17739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } }
I20260811 02:57:41.328907 17739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:41.330006 17627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:41.339767 17750 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:41.339958 17750 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 02:57:41.554505 17576 tablet.cc:2369] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:41.572997 17724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:41.573413 17724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } }
I20260811 02:57:41.575307 17724 leader_election.cc:288] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17fb94eb8b7b4caea6f9669c114504f4 (127.13.94.190:41293)
I20260811 02:57:41.576547 17600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "91d3a9f0f5474fb9b0409772125153e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fb94eb8b7b4caea6f9669c114504f4" is_pre_election: true
I20260811 02:57:41.577207 17600 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91d3a9f0f5474fb9b0409772125153e2 in term 0.
I20260811 02:57:41.577956 17656 leader_election.cc:302] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 17fb94eb8b7b4caea6f9669c114504f4, 91d3a9f0f5474fb9b0409772125153e2; no voters:
I20260811 02:57:41.578787 17724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:41.578966 17724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:41.579090 17724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:41.582428 17724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } }
I20260811 02:57:41.583639 17600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "91d3a9f0f5474fb9b0409772125153e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fb94eb8b7b4caea6f9669c114504f4"
I20260811 02:57:41.583875 17600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:41.587463 17600 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91d3a9f0f5474fb9b0409772125153e2 in term 1.
I20260811 02:57:41.587977 17656 leader_election.cc:302] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 17fb94eb8b7b4caea6f9669c114504f4, 91d3a9f0f5474fb9b0409772125153e2; no voters:
I20260811 02:57:41.590610 17724 leader_election.cc:288] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [CANDIDATE]: Term 1 election: Requested vote from peers 17fb94eb8b7b4caea6f9669c114504f4 (127.13.94.190:41293)
I20260811 02:57:41.590843 17724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:41.590991 17724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 1 LEADER]: Becoming Leader. State: Replica: 91d3a9f0f5474fb9b0409772125153e2, State: Running, Role: LEADER
I20260811 02:57:41.591535 17724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } }
I20260811 02:57:41.600765 17724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91d3a9f0f5474fb9b0409772125153e2. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } }
I20260811 02:57:41.601085 17724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:41.601975 17754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:41.604611 17754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:41.613584 17600 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Refusing update from remote peer 91d3a9f0f5474fb9b0409772125153e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:41.614683 17752 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:41.621987 17739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91d3a9f0f5474fb9b0409772125153e2. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } }
I20260811 02:57:41.622270 17739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:41.625017 17752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } }
I20260811 02:57:41.625289 17752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:41.627182 17739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } }
I20260811 02:57:41.627458 17754 catalog_manager.cc:1347] Generated new cluster ID: 7c9307eb863047528c0daea25d8f82d8
I20260811 02:57:41.627462 17739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:41.627564 17754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:41.659430 17754 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:41.661232 17754 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:41.672224 17754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: Generated new TSK 0
I20260811 02:57:41.673512 17754 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.94.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.94.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.129.keytab.
Entry for principal kudu/127.13.94.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.129.keytab.
I20260811 02:57:41.761781 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:0
--local_ip_for_outbound_sockets=127.13.94.129
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:41293,127.13.94.189:40361
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.129.keytab
--principal=kudu/127.13.94.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.94.148:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:42.115295 17762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:42.115603 17762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:42.115768 17762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:42.123517 17762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:42.123795 17762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:57:42.134604 17762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.129.keytab
--principal=kudu/127.13.94.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:41293,127.13.94.189:40361
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:42.136070 17762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:42.137665 17762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:42.152750 17768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:42.152930 17767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:42.153280 17770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:42.154357 17762 server_base.cc:1034] running on GCE node
I20260811 02:57:42.155258 17762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:42.156443 17762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:42.157781 17762 hybrid_clock.cc:648] HybridClock initialized: now 1786417062157693 us; error 48 us; skew 500 ppm
I20260811 02:57:42.158143 17762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 02:57:42 dist-test-slave-p3t5 krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:42.161881 17762 init.cc:377] Logged in from keytab as kudu/127.13.94.129@KRBTEST.COM (short username kudu)
I20260811 02:57:42.163913 17762 webserver.cc:460] Webserver started at http://127.13.94.129:36149/ using document root <none> and password file <none>
I20260811 02:57:42.164700 17762 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:42.164794 17762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:42.165135 17762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:42.168172 17762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/instance:
uuid: "e9aa9c71f0b44978856df6dbdc070eb9"
format_stamp: "Formatted at 2026-08-11 02:57:42 on dist-test-slave-p3t5"
I20260811 02:57:42.169020 17762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance:
uuid: "e9aa9c71f0b44978856df6dbdc070eb9"
format_stamp: "Formatted at 2026-08-11 02:57:42 on dist-test-slave-p3t5"
I20260811 02:57:42.175130 17762 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 02:57:42.179105 17778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:42.180788 17762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:42.181031 17762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "e9aa9c71f0b44978856df6dbdc070eb9"
format_stamp: "Formatted at 2026-08-11 02:57:42 on dist-test-slave-p3t5"
I20260811 02:57:42.181306 17762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:42.198213 17762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:42.203078 17762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:42.203404 17762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:42.204507 17762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:42.206020 17762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:42.206151 17762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:42.206295 17762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:42.206393 17762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:42.242450 17762 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:33703
I20260811 02:57:42.242543 17906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:33703 every 8 connection(s)
I20260811 02:57:42.243865 17762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:42.247381 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 17762
I20260811 02:57:42.247632 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance
Aug 11 02:57:42 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.129@KRBTEST.COM for kudu/127.13.94.189@KRBTEST.COM
Aug 11 02:57:42 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.129@KRBTEST.COM for kudu/127.13.94.190@KRBTEST.COM
I20260811 02:57:42.273049 17907 heartbeater.cc:347] Connected to a master server at 127.13.94.189:40361
I20260811 02:57:42.273128 17908 heartbeater.cc:347] Connected to a master server at 127.13.94.190:41293
I20260811 02:57:42.273416 17908 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:42.273419 17907 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:42.274461 17908 heartbeater.cc:511] Master 127.13.94.190:41293 requested a full tablet report, sending...
I20260811 02:57:42.274482 17907 heartbeater.cc:511] Master 127.13.94.189:40361 requested a full tablet report, sending...
I20260811 02:57:42.276611 17671 ts_manager.cc:194] Registered new tserver with Master: e9aa9c71f0b44978856df6dbdc070eb9 (127.13.94.129:33703)
I20260811 02:57:42.276636 17589 ts_manager.cc:194] Registered new tserver with Master: e9aa9c71f0b44978856df6dbdc070eb9 (127.13.94.129:33703)
I20260811 02:57:42.278244 17671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.94.129@KRBTEST.COM'} at 127.13.94.129:33149
I20260811 02:57:42.283555 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:42.289235 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:42.322410 17671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50096:
name: "first_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:42.342170 17750 catalog_manager.cc:1259] Loaded cluster ID: 7c9307eb863047528c0daea25d8f82d8
I20260811 02:57:42.342324 17750 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: loading cluster ID for follower catalog manager: success
I20260811 02:57:42.345274 17750 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: acquiring CA information for follower catalog manager: success
I20260811 02:57:42.348097 17750 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:42.360692 17836 tablet_service.cc:1467] Processing CreateTablet for tablet bd12ba4ba8ea4deebb7b059546592ce4 (DEFAULT_TABLE table=first_table [id=7fe9c6d5429d444d82a87d5f32fa61ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:42.362134 17836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd12ba4ba8ea4deebb7b059546592ce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:42.374074 17929 tablet_bootstrap.cc:492] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Bootstrap starting.
I20260811 02:57:42.377190 17929 tablet_bootstrap.cc:654] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:42.378461 17929 log.cc:824] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:42.381163 17929 tablet_bootstrap.cc:492] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: No bootstrap required, opened a new log
I20260811 02:57:42.381464 17929 ts_tablet_manager.cc:1397] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 02:57:42.388969 17929 raft_consensus.cc:348] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 33703 } }
I20260811 02:57:42.389397 17929 raft_consensus.cc:374] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:42.389523 17929 raft_consensus.cc:729] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Initialized, Role: FOLLOWER
I20260811 02:57:42.390230 17929 consensus_queue.cc:260] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 33703 } }
I20260811 02:57:42.390496 17929 raft_consensus.cc:388] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:42.390658 17929 raft_consensus.cc:482] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:42.390830 17929 raft_consensus.cc:3037] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:42.392974 17929 raft_consensus.cc:504] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 33703 } }
I20260811 02:57:42.393558 17929 leader_election.cc:302] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9aa9c71f0b44978856df6dbdc070eb9; no voters:
I20260811 02:57:42.394284 17929 leader_election.cc:288] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:42.394474 17932 raft_consensus.cc:2781] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:42.395200 17932 raft_consensus.cc:686] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 LEADER]: Becoming Leader. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Running, Role: LEADER
I20260811 02:57:42.395483 17907 heartbeater.cc:503] Master 127.13.94.189:40361 was elected leader, sending a full tablet report...
I20260811 02:57:42.395861 17932 consensus_queue.cc:237] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 33703 } }
I20260811 02:57:42.396343 17929 ts_tablet_manager.cc:1428] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260811 02:57:42.402486 17671 catalog_manager.cc:5495] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9aa9c71f0b44978856df6dbdc070eb9 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 33703 } health_report { overall_health: HEALTHY } } }
WARNING: no policy specified for kudu/127.13.94.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.94.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.188.keytab.
Entry for principal kudu/127.13.94.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.188.keytab.
Aug 11 02:57:43 dist-test-slave-p3t5 krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 02:57:43 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.188@KRBTEST.COM for kudu/127.13.94.190@KRBTEST.COM
Aug 11 02:57:43 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.188@KRBTEST.COM for kudu/127.13.94.189@KRBTEST.COM
Aug 11 02:57:43 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.188@KRBTEST.COM for kudu/127.13.94.188@KRBTEST.COM
Aug 11 02:57:43 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.94.188@KRBTEST.COM
Aug 11 02:57:43 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417061, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.189@KRBTEST.COM for kudu/127.13.94.188@KRBTEST.COM
I20260811 02:57:43.574337 17671 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.94.188:34185: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c27799f097aa48f78a57a79e2c614191"
member_type: NON_VOTER
last_known_addr {
host: "127.13.94.188"
port: 34185
}
attrs {
promote: true
}
}
dest_uuid: "91d3a9f0f5474fb9b0409772125153e2"
cas_config_opid_index: -1
I20260811 02:57:43.575840 17671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: true } }
I20260811 02:57:43.583305 17600 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Refusing update from remote peer 91d3a9f0f5474fb9b0409772125153e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 02:57:43.584039 17924 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 02:57:43.586509 17924 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:43.590806 18057 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index -1 to 8, NON_VOTER c27799f097aa48f78a57a79e2c614191 (127.13.94.188) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: true } } }
I20260811 02:57:43.591475 17600 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index -1 to 8, NON_VOTER c27799f097aa48f78a57a79e2c614191 (127.13.94.188) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: true } } }
I20260811 02:57:43.595502 18057 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.94.188:34185
I20260811 02:57:43.596065 17924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: true } } }
I20260811 02:57:43.596390 17924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:43.602933 17924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: true } } }
I20260811 02:57:43.603339 17924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:43.613022 18059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: true } } }
I20260811 02:57:43.613592 18059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:43.642457 18057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: NON_VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: true } } }
I20260811 02:57:43.642781 18057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:43.650061 17924 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: attempting to promote NON_VOTER c27799f097aa48f78a57a79e2c614191 to VOTER
I20260811 02:57:43.651070 17924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:43.654367 17600 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Refusing update from remote peer 91d3a9f0f5474fb9b0409772125153e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 02:57:43.655557 18057 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 02:57:43.659703 18057 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 02:57:43.664136 18077 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c27799f097aa48f78a57a79e2c614191 (127.13.94.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:43.668605 18057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:43.668888 18057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:43.669214 18057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:43.669422 18057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:43.666925 17600 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c27799f097aa48f78a57a79e2c614191 (127.13.94.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:43.676407 17924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:43.676889 17924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:43.677793 18059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:43.678115 18059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: FOLLOWER
WARNING: no policy specified for kudu/127.13.94.188@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.13.94.188@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.188 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.188.keytab.
Entry for principal kudu/127.13.94.188 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.188.keytab.
W20260811 02:57:44.285943 17653 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.94.188:34185: connect: Connection refused (error 111)
W20260811 02:57:44.287863 17653 consensus_peers.cc:487] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 -> Peer c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185): Couldn't send request to peer c27799f097aa48f78a57a79e2c614191. Status: Network error: Client connection negotiation failed: client connection to 127.13.94.188:34185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 02:57:44.775925 18093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:44.776422 18093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: LEADER
Aug 11 02:57:44 dist-test-slave-p3t5 krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417064, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 02:57:45.091907 17671 master_service.cc:291] Master 127.13.94.188:34185 already present
WARNING: no policy specified for kudu/127.13.94.190@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.13.94.190@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.190 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab.
Entry for principal kudu/127.13.94.190 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab.
I20260811 02:57:45.933993 18109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:45.934505 18109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: LEADER
W20260811 02:57:45.945721 17653 consensus_peers.cc:487] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 -> Peer c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185): Couldn't send request to peer c27799f097aa48f78a57a79e2c614191. Status: Network error: Client connection negotiation failed: client connection to 127.13.94.188:34185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 02:57:46.166220 13690 dynamic_multi_master-test.cc:554] Add master stderr: W20260811 02:57:46.017119 18239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:46.017462 18239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:46.017572 18239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:46.025087 18239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:46.025224 18239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:46.025277 18239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:46.025327 18239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:57:46.025408 18239 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Illegal state: Failed bringing up new master. See Kudu Master log for details.: Master 127.13.94.190:41293 already running
I20260811 02:57:46.169458 13690 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:46.169687 13690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:46.172533 13690 dynamic_multi_master-test.cc:645] Shutting down the old cluster
I20260811 02:57:46.172762 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 17762
Aug 11 02:57:46 dist-test-slave-p3t5 krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.94.129@KRBTEST.COM
I20260811 02:57:46.191315 17901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:46.285960 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 17762
I20260811 02:57:46.307788 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 17553
I20260811 02:57:46.309053 17622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:46.425542 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 17553
I20260811 02:57:46.451995 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 17634
I20260811 02:57:46.453125 17708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:57:46.578941 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 17634
I20260811 02:57:46.578948 18241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91d3a9f0f5474fb9b0409772125153e2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
2026-08-11T02:57:46Z chronyd exiting
I20260811 02:57:46.619051 13690 dynamic_multi_master-test.cc:648] Bringing up the migrated cluster
Aug 11 02:57:46 dist-test-slave-p3t5 krb5kdc[18268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:46 dist-test-slave-p3t5 krb5kdc[18268](info): set up 2 sockets
Aug 11 02:57:46 dist-test-slave-p3t5 krb5kdc[18268](info): commencing operation
krb5kdc: starting...
W20260811 02:57:47.733736 13690 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/test-user.keytab.
Aug 11 02:57:47 dist-test-slave-p3t5 krb5kdc[18268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.94.190@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.13.94.190@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.190 with kvno 4, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab.
Entry for principal kudu/127.13.94.190 with kvno 4, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab.
I20260811 02:57:47.945241 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:41293
--webserver_interface=127.13.94.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab
--principal=kudu/127.13.94.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.94.148:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:41293,127.13.94.189:40361,127.13.94.188:34185
--master_auto_join_cluster=false with env {KRB5CCNAME=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:48.301733 18281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:48.302192 18281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:48.302357 18281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:48.309860 18281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:48.310070 18281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:48.310173 18281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:48.310279 18281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:48.320712 18281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:41293,127.13.94.189:40361,127.13.94.188:34185
--master_auto_join_cluster=false
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab
--principal=kudu/127.13.94.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:41293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:48.322343 18281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:48.323799 18281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:48.336500 18286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:48.336440 18287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:48.336464 18289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:48.337832 18281 server_base.cc:1034] running on GCE node
I20260811 02:57:48.338740 18281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:48.340305 18281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:48.341586 18281 hybrid_clock.cc:648] HybridClock initialized: now 1786417068341506 us; error 51 us; skew 500 ppm
I20260811 02:57:48.342027 18281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 02:57:48 dist-test-slave-p3t5 krb5kdc[18268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417068, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:48.345590 18281 init.cc:377] Logged in from keytab as kudu/127.13.94.190@KRBTEST.COM (short username kudu)
I20260811 02:57:48.347306 18281 webserver.cc:460] Webserver started at http://127.13.94.190:38271/ using document root <none> and password file <none>
I20260811 02:57:48.348084 18281 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:48.348245 18281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:48.354169 18281 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:57:48.357115 18296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:48.358618 18281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:57:48.358821 18281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "17fb94eb8b7b4caea6f9669c114504f4"
format_stamp: "Formatted at 2026-08-11 02:57:40 on dist-test-slave-p3t5"
I20260811 02:57:48.359783 18281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:48.376377 18281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:48.380697 18281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:48.381049 18281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:48.402565 18281 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:41293
I20260811 02:57:48.402585 18357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:41293 every 8 connection(s)
I20260811 02:57:48.404306 18281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:48.411960 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 18281
I20260811 02:57:48.412271 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/wal/instance
I20260811 02:57:48.412632 18358 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:48.419109 18358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: Bootstrap starting.
WARNING: no policy specified for kudu/127.13.94.189@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.13.94.189@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260811 02:57:48.441939 18358 log.cc:824] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:48.455698 18358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:48.456408 18358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: Bootstrap complete.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.189 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab.
Entry for principal kudu/127.13.94.189 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab.
I20260811 02:57:48.469220 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:40361
--webserver_interface=127.13.94.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab
--principal=kudu/127.13.94.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.94.148:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:41293,127.13.94.189:40361,127.13.94.188:34185
--master_auto_join_cluster=false with env {KRB5CCNAME=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:57:48.468277 18358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:48.469936 18358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fb94eb8b7b4caea6f9669c114504f4, State: Initialized, Role: FOLLOWER
I20260811 02:57:48.470880 18358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:48.471912 18364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:48.472306 18364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:48.473664 18358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:48.485695 18376 catalog_manager.cc:1259] Loaded cluster ID: 7c9307eb863047528c0daea25d8f82d8
I20260811 02:57:48.485831 18376 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: loading cluster ID for follower catalog manager: success
I20260811 02:57:48.490926 18376 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: acquiring CA information for follower catalog manager: success
I20260811 02:57:48.494006 18376 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 02:57:48.845157 18373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:48.845558 18373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:48.845742 18373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:48.854014 18373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:48.854219 18373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:48.854316 18373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:48.854431 18373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:48.865302 18373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:41293,127.13.94.189:40361,127.13.94.188:34185
--master_auto_join_cluster=false
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab
--principal=kudu/127.13.94.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:40361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:48.866906 18373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:48.868384 18373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:48.882043 18382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:48.882151 18383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:48.881924 18385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:48.883204 18373 server_base.cc:1034] running on GCE node
I20260811 02:57:48.884193 18373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:48.885977 18373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:48.887231 18373 hybrid_clock.cc:648] HybridClock initialized: now 1786417068887084 us; error 122 us; skew 500 ppm
I20260811 02:57:48.887567 18373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 02:57:48 dist-test-slave-p3t5 krb5kdc[18268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417068, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:48.891307 18373 init.cc:377] Logged in from keytab as kudu/127.13.94.189@KRBTEST.COM (short username kudu)
I20260811 02:57:48.893131 18373 webserver.cc:460] Webserver started at http://127.13.94.189:37617/ using document root <none> and password file <none>
I20260811 02:57:48.894059 18373 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:48.894184 18373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:48.900656 18373 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:57:48.904130 18393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:48.905797 18373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:57:48.906000 18373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "91d3a9f0f5474fb9b0409772125153e2"
format_stamp: "Formatted at 2026-08-11 02:57:41 on dist-test-slave-p3t5"
I20260811 02:57:48.907064 18373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:48.928618 18373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:48.933535 18373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:48.933897 18373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:48.954309 18373 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:40361
I20260811 02:57:48.954336 18444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:40361 every 8 connection(s)
I20260811 02:57:48.956066 18373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:57:48.959865 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 18373
I20260811 02:57:48.960080 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/wal/instance
I20260811 02:57:48.963501 18445 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:48.967375 18445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: Bootstrap starting.
WARNING: no policy specified for kudu/127.13.94.188@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.13.94.188@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260811 02:57:48.989192 18445 log.cc:824] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:49.003104 18445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:49.004737 18445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: Bootstrap complete.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.188 with kvno 4, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.188.keytab.
Entry for principal kudu/127.13.94.188 with kvno 4, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.188.keytab.
I20260811 02:57:49.014299 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:34185
--webserver_interface=127.13.94.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.188.keytab
--principal=kudu/127.13.94.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.94.148:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:41293,127.13.94.189:40361,127.13.94.188:34185
--master_auto_join_cluster=false with env {KRB5CCNAME=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:57:49.017491 18445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:49.018707 18445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91d3a9f0f5474fb9b0409772125153e2, State: Initialized, Role: FOLLOWER
I20260811 02:57:49.019598 18445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:49.020658 18450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:49.021193 18450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:49.022055 18445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:49.033303 18464 catalog_manager.cc:1259] Loaded cluster ID: 7c9307eb863047528c0daea25d8f82d8
I20260811 02:57:49.033433 18464 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: loading cluster ID for follower catalog manager: success
I20260811 02:57:49.037698 18464 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: acquiring CA information for follower catalog manager: success
I20260811 02:57:49.040369 18464 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 02:57:49.383008 18451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:49.383411 18451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:49.383579 18451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:49.391475 18451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:49.391670 18451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:49.391767 18451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:49.391867 18451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:49.403632 18451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:41293,127.13.94.189:40361,127.13.94.188:34185
--master_auto_join_cluster=false
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.188.keytab
--principal=kudu/127.13.94.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:34185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.94.188
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:49.405102 18451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:49.406620 18451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:49.419543 18474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:49.419535 18471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:49.419585 18472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:49.421010 18451 server_base.cc:1034] running on GCE node
I20260811 02:57:49.422076 18451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:49.423830 18451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:49.425138 18451 hybrid_clock.cc:648] HybridClock initialized: now 1786417069425056 us; error 54 us; skew 500 ppm
I20260811 02:57:49.425522 18451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 02:57:49 dist-test-slave-p3t5 krb5kdc[18268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417069, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:49.429031 18451 init.cc:377] Logged in from keytab as kudu/127.13.94.188@KRBTEST.COM (short username kudu)
I20260811 02:57:49.430630 18451 webserver.cc:460] Webserver started at http://127.13.94.188:41171/ using document root <none> and password file <none>
I20260811 02:57:49.431352 18451 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:49.431447 18451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:49.437099 18451 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:57:49.440201 18482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:49.441785 18451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 02:57:49.441977 18451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/wal
uuid: "c27799f097aa48f78a57a79e2c614191"
format_stamp: "Formatted at 2026-08-11 02:57:43 on dist-test-slave-p3t5"
I20260811 02:57:49.442920 18451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:49.476404 18451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:49.481333 18451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:49.481735 18451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:49.503453 18451 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.188:34185
I20260811 02:57:49.503473 18539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.188:34185 every 8 connection(s)
I20260811 02:57:49.505170 18451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
I20260811 02:57:49.512885 18540 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:49.513980 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 18451
I20260811 02:57:49.514200 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/wal/instance
I20260811 02:57:49.517802 18540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191: Bootstrap starting.
Aug 11 02:57:49 dist-test-slave-p3t5 krb5kdc[18268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.94.190@KRBTEST.COM
I20260811 02:57:49.543128 18540 log.cc:824] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:57:49 dist-test-slave-p3t5 krb5kdc[18268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.94.189@KRBTEST.COM
I20260811 02:57:49.558523 18540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:49.559294 18540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191: Bootstrap complete.
I20260811 02:57:49.571749 18540 raft_consensus.cc:348] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:49.573088 18540 raft_consensus.cc:729] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27799f097aa48f78a57a79e2c614191, State: Initialized, Role: FOLLOWER
I20260811 02:57:49.574067 18540 consensus_queue.cc:260] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:49.575160 18547 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:49.575600 18547 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:49.576473 18540 sys_catalog.cc:564] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 02:57:49 dist-test-slave-p3t5 krb5kdc[18268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.94.188@KRBTEST.COM
I20260811 02:57:49.588810 18558 catalog_manager.cc:1259] Loaded cluster ID: 7c9307eb863047528c0daea25d8f82d8
I20260811 02:57:49.588932 18558 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191: loading cluster ID for follower catalog manager: success
I20260811 02:57:49.592962 18558 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191: acquiring CA information for follower catalog manager: success
I20260811 02:57:49.595909 18558 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:57:50.145555 18561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:50.145975 18561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:50.148561 18561 leader_election.cc:288] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91d3a9f0f5474fb9b0409772125153e2 (127.13.94.189:40361), c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185)
I20260811 02:57:50.168948 18515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "c27799f097aa48f78a57a79e2c614191" is_pre_election: true
I20260811 02:57:50.168949 18420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "91d3a9f0f5474fb9b0409772125153e2" is_pre_election: true
I20260811 02:57:50.169576 18420 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fb94eb8b7b4caea6f9669c114504f4 in term 1.
I20260811 02:57:50.169579 18515 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fb94eb8b7b4caea6f9669c114504f4 in term 1.
I20260811 02:57:50.170500 18299 leader_election.cc:302] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 17fb94eb8b7b4caea6f9669c114504f4, 91d3a9f0f5474fb9b0409772125153e2, c27799f097aa48f78a57a79e2c614191; no voters:
I20260811 02:57:50.171012 18561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 02:57:50.171201 18561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:50.171332 18561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:50.176407 18561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:50.177278 18561 leader_election.cc:288] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 2 election: Requested vote from peers 91d3a9f0f5474fb9b0409772125153e2 (127.13.94.189:40361), c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185)
I20260811 02:57:50.178054 18515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "c27799f097aa48f78a57a79e2c614191"
I20260811 02:57:50.178053 18420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "91d3a9f0f5474fb9b0409772125153e2"
I20260811 02:57:50.178331 18420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:50.178331 18515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:50.184465 18515 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fb94eb8b7b4caea6f9669c114504f4 in term 2.
I20260811 02:57:50.184461 18420 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fb94eb8b7b4caea6f9669c114504f4 in term 2.
I20260811 02:57:50.185081 18299 leader_election.cc:302] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 17fb94eb8b7b4caea6f9669c114504f4, 91d3a9f0f5474fb9b0409772125153e2, c27799f097aa48f78a57a79e2c614191; no voters:
I20260811 02:57:50.185513 18561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:57:50.186084 18561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 2 LEADER]: Becoming Leader. State: Replica: 17fb94eb8b7b4caea6f9669c114504f4, State: Running, Role: LEADER
I20260811 02:57:50.186630 18561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:50.189800 18566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17fb94eb8b7b4caea6f9669c114504f4. Latest consensus state: current_term: 2 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:50.190230 18566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:50.191062 18568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:50.193437 18568 catalog_manager.cc:669] Loaded metadata for table first_table [id=7fe9c6d5429d444d82a87d5f32fa61ea]
I20260811 02:57:50.195163 18568 tablet_loader.cc:96] loaded metadata for tablet bd12ba4ba8ea4deebb7b059546592ce4 (table first_table [id=7fe9c6d5429d444d82a87d5f32fa61ea])
I20260811 02:57:50.195425 18568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:50.196220 18568 catalog_manager.cc:1259] Loaded cluster ID: 7c9307eb863047528c0daea25d8f82d8
I20260811 02:57:50.196295 18568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:50.197404 18568 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:50.198390 18568 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: Loaded TSK: 0
I20260811 02:57:50.199393 18568 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.94.129@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.13.94.129@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.129 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.129.keytab.
Entry for principal kudu/127.13.94.129 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.129.keytab.
I20260811 02:57:50.257129 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:0
--local_ip_for_outbound_sockets=127.13.94.129
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:41293,127.13.94.189:40361,127.13.94.188:34185
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.129.keytab
--principal=kudu/127.13.94.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.94.148:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:50.622889 18571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:50.623292 18571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:50.623445 18571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:50.631361 18571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:50.631603 18571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:57:50.641980 18571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.129.keytab
--principal=kudu/127.13.94.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:41293,127.13.94.189:40361,127.13.94.188:34185
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:57:50.643471 18571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:50.644907 18571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:50.659682 18515 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 2 FOLLOWER]: Refusing update from remote peer 17fb94eb8b7b4caea6f9669c114504f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
W20260811 02:57:50.659590 18576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:50.659650 18577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:50.660396 18579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:50.661120 18571 server_base.cc:1034] running on GCE node
I20260811 02:57:50.661012 18566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 02:57:50.661998 18571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:50.663589 18571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:50.664844 18571 hybrid_clock.cc:648] HybridClock initialized: now 1786417070664716 us; error 90 us; skew 500 ppm
I20260811 02:57:50.665218 18571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:57:50.665536 18581 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17fb94eb8b7b4caea6f9669c114504f4. Latest consensus state: current_term: 2 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:50.666059 18581 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:50.667687 18566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:50.667987 18566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:50.669150 18420 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 2 FOLLOWER]: Refusing update from remote peer 17fb94eb8b7b4caea6f9669c114504f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20260811 02:57:50.670295 18566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
Aug 11 02:57:50 dist-test-slave-p3t5 krb5kdc[18268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:50.675297 18571 init.cc:377] Logged in from keytab as kudu/127.13.94.129@KRBTEST.COM (short username kudu)
I20260811 02:57:50.681295 18581 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:50.682098 18581 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:50.688457 18571 webserver.cc:460] Webserver started at http://127.13.94.129:35935/ using document root <none> and password file <none>
I20260811 02:57:50.689682 18571 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:50.689883 18571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:50.690104 18595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:50.690614 18595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:50.690836 18588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17fb94eb8b7b4caea6f9669c114504f4. Latest consensus state: current_term: 2 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:50.691157 18588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:50.696478 18571 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:57:50.700341 18597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:50.701980 18571 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:57:50.702198 18571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "e9aa9c71f0b44978856df6dbdc070eb9"
format_stamp: "Formatted at 2026-08-11 02:57:42 on dist-test-slave-p3t5"
I20260811 02:57:50.703226 18571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:50.718876 18571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:50.724175 18571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:50.724525 18571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:50.725772 18571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:50.727788 18604 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 02:57:50.733482 18571 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:57:50.733651 18571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 02:57:50.733920 18571 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:57:50.737556 18571 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:57:50.737748 18571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.002s sys 0.002s
I20260811 02:57:50.738130 18604 tablet_bootstrap.cc:492] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Bootstrap starting.
I20260811 02:57:50.764431 18604 log.cc:824] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:50.770956 18604 tablet_bootstrap.cc:492] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:50.771843 18604 tablet_bootstrap.cc:492] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Bootstrap complete.
I20260811 02:57:50.774024 18604 ts_tablet_manager.cc:1397] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Time spent bootstrapping tablet: real 0.036s user 0.020s sys 0.014s
I20260811 02:57:50.781260 18571 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:35351
I20260811 02:57:50.781283 18724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:35351 every 8 connection(s)
I20260811 02:57:50.783113 18571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:50.784474 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 18571
I20260811 02:57:50.784723 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:50.808826 18604 raft_consensus.cc:348] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 33703 } }
I20260811 02:57:50.810544 18604 raft_consensus.cc:729] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Initialized, Role: FOLLOWER
I20260811 02:57:50.811448 18604 consensus_queue.cc:260] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 33703 } }
I20260811 02:57:50.811784 18604 raft_consensus.cc:388] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:50.811961 18604 raft_consensus.cc:482] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:50.812152 18604 raft_consensus.cc:3037] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:50.818720 18604 raft_consensus.cc:504] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 33703 } }
I20260811 02:57:50.819471 18604 leader_election.cc:302] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9aa9c71f0b44978856df6dbdc070eb9; no voters:
Aug 11 02:57:50 dist-test-slave-p3t5 krb5kdc[18268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.129@KRBTEST.COM for kudu/127.13.94.188@KRBTEST.COM
Aug 11 02:57:50 dist-test-slave-p3t5 krb5kdc[18268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.129@KRBTEST.COM for kudu/127.13.94.189@KRBTEST.COM
I20260811 02:57:50.821172 18604 leader_election.cc:288] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:57:50.821332 18738 raft_consensus.cc:2781] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 2 FOLLOWER]: Leader election won for term 2
Aug 11 02:57:50 dist-test-slave-p3t5 krb5kdc[18268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.129@KRBTEST.COM for kudu/127.13.94.190@KRBTEST.COM
I20260811 02:57:50.835652 18726 heartbeater.cc:347] Connected to a master server at 127.13.94.188:34185
I20260811 02:57:50.836160 18726 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:50.837137 18738 raft_consensus.cc:686] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [term 2 LEADER]: Becoming Leader. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Running, Role: LEADER
I20260811 02:57:50.837347 18726 heartbeater.cc:511] Master 127.13.94.188:34185 requested a full tablet report, sending...
I20260811 02:57:50.837837 18738 consensus_queue.cc:237] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 33703 } }
I20260811 02:57:50.840368 18727 heartbeater.cc:347] Connected to a master server at 127.13.94.190:41293
I20260811 02:57:50.840564 18727 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:50.841148 18727 heartbeater.cc:511] Master 127.13.94.190:41293 requested a full tablet report, sending...
I20260811 02:57:50.841195 18604 ts_tablet_manager.cc:1428] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9: Time spent starting tablet: real 0.067s user 0.034s sys 0.014s
I20260811 02:57:50.844344 18731 heartbeater.cc:347] Connected to a master server at 127.13.94.189:40361
I20260811 02:57:50.844507 18731 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:50.845000 18731 heartbeater.cc:511] Master 127.13.94.189:40361 requested a full tablet report, sending...
I20260811 02:57:50.845371 18503 ts_manager.cc:194] Registered new tserver with Master: e9aa9c71f0b44978856df6dbdc070eb9 (127.13.94.129:35351)
I20260811 02:57:50.846479 18314 ts_manager.cc:194] Registered new tserver with Master: e9aa9c71f0b44978856df6dbdc070eb9 (127.13.94.129:35351)
I20260811 02:57:50.847867 18410 ts_manager.cc:194] Registered new tserver with Master: e9aa9c71f0b44978856df6dbdc070eb9 (127.13.94.129:35351)
I20260811 02:57:50.848132 18314 catalog_manager.cc:5495] T bd12ba4ba8ea4deebb7b059546592ce4 P e9aa9c71f0b44978856df6dbdc070eb9 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 33703 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:50.854974 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:57:50.858830 18561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:50.859189 18561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:50.866698 18314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.94.129@KRBTEST.COM'} at 127.13.94.129:35151
I20260811 02:57:50.870347 18727 heartbeater.cc:503] Master 127.13.94.190:41293 was elected leader, sending a full tablet report...
I20260811 02:57:50.884945 13690 dynamic_multi_master-test.cc:680] Transferring leadership to master: c27799f097aa48f78a57a79e2c614191
I20260811 02:57:50.889101 18326 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "17fb94eb8b7b4caea6f9669c114504f4"
mode: GRACEFUL
new_leader_uuid: "c27799f097aa48f78a57a79e2c614191"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48148
I20260811 02:57:50.889523 18326 raft_consensus.cc:595] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 2 LEADER]: Received request to transfer leadership to TS c27799f097aa48f78a57a79e2c614191
I20260811 02:57:51.390969 18566 raft_consensus.cc:979] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: : Instructing follower c27799f097aa48f78a57a79e2c614191 to start an election
I20260811 02:57:51.391131 18566 raft_consensus.cc:1067] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 2 LEADER]: Signalling peer c27799f097aa48f78a57a79e2c614191 to start an election
I20260811 02:57:51.392233 18515 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c27799f097aa48f78a57a79e2c614191"
from {username='kudu', principal='kudu/127.13.94.190@KRBTEST.COM'} at 127.0.0.1:54838
I20260811 02:57:51.392536 18515 raft_consensus.cc:482] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 02:57:51.392694 18515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:51.394922 18515 raft_consensus.cc:504] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:51.397171 18515 leader_election.cc:288] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [CANDIDATE]: Term 3 election: Requested vote from peers 17fb94eb8b7b4caea6f9669c114504f4 (127.13.94.190:41293), 91d3a9f0f5474fb9b0409772125153e2 (127.13.94.189:40361)
I20260811 02:57:51.406499 18326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c27799f097aa48f78a57a79e2c614191" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "17fb94eb8b7b4caea6f9669c114504f4"
I20260811 02:57:51.406971 18326 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 2 LEADER]: Stepping down as leader of term 2
I20260811 02:57:51.407115 18326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 17fb94eb8b7b4caea6f9669c114504f4, State: Running, Role: LEADER
I20260811 02:57:51.407212 18420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c27799f097aa48f78a57a79e2c614191" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "91d3a9f0f5474fb9b0409772125153e2"
I20260811 02:57:51.407434 18420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:51.407538 18326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:51.408107 18326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:51.410437 18420 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c27799f097aa48f78a57a79e2c614191 in term 3.
I20260811 02:57:51.410805 18326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c27799f097aa48f78a57a79e2c614191 in term 3.
I20260811 02:57:51.411346 18487 leader_election.cc:302] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91d3a9f0f5474fb9b0409772125153e2, c27799f097aa48f78a57a79e2c614191; no voters:
I20260811 02:57:51.412217 18752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:51.412905 18752 raft_consensus.cc:686] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 3 LEADER]: Becoming Leader. State: Replica: c27799f097aa48f78a57a79e2c614191, State: Running, Role: LEADER
I20260811 02:57:51.413446 18752 consensus_queue.cc:237] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:51.415131 18753 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c27799f097aa48f78a57a79e2c614191. Latest consensus state: current_term: 3 leader_uuid: "c27799f097aa48f78a57a79e2c614191" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:51.415438 18753 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:51.416271 18754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:51.418915 18754 catalog_manager.cc:669] Loaded metadata for table first_table [id=7fe9c6d5429d444d82a87d5f32fa61ea]
I20260811 02:57:51.420656 18754 tablet_loader.cc:96] loaded metadata for tablet bd12ba4ba8ea4deebb7b059546592ce4 (table first_table [id=7fe9c6d5429d444d82a87d5f32fa61ea])
I20260811 02:57:51.420992 18754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:51.422031 18754 catalog_manager.cc:1259] Loaded cluster ID: 7c9307eb863047528c0daea25d8f82d8
I20260811 02:57:51.422108 18754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:51.423571 18754 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:51.424484 18754 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191: Loaded TSK: 0
I20260811 02:57:51.425554 18754 catalog_manager.cc:1514] Initializing in-progress tserver states...
Aug 11 02:57:51 dist-test-slave-p3t5 krb5kdc[18268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.94.129@KRBTEST.COM
I20260811 02:57:51.628577 18654 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
17fb94eb8b7b4caea6f9669c114504f4 | 127.13.94.190:41293 | HEALTHY
91d3a9f0f5474fb9b0409772125153e2 | 127.13.94.189:40361 | HEALTHY
c27799f097aa48f78a57a79e2c614191 | 127.13.94.188:34185 | HEALTHY
All reported replicas are:
A = 17fb94eb8b7b4caea6f9669c114504f4
B = 91d3a9f0f5474fb9b0409772125153e2
C = c27799f097aa48f78a57a79e2c614191
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
A | A B C | 3 | 9 | Yes
B | A B C | 3 | 9 | Yes
C | A B C* | 3 | 9 | Yes
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 3 server(s) checked
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_reuseport | true | experimental | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb | hidden | 127.13.94.190:41293
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb | hidden | 127.13.94.189:40361
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb | hidden | 127.13.94.188:34185
tsk_num_rsa_bits | 512 | experimental | all 3 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.13.94.148:43463 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
e9aa9c71f0b44978856df6dbdc070eb9 | 127.13.94.129:35351 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.13.94.129 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.13.94.148:43463 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 3
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: master consensus error: there are master consensus conflicts
FAILED
I20260811 02:57:51.664681 13690 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 02:57:51.868206 18420 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 3 FOLLOWER]: Refusing update from remote peer c27799f097aa48f78a57a79e2c614191: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20260811 02:57:51.869230 18753 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260811 02:57:51.876729 18805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c27799f097aa48f78a57a79e2c614191. Latest consensus state: current_term: 3 leader_uuid: "c27799f097aa48f78a57a79e2c614191" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:51.877213 18805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:51.880947 18805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c27799f097aa48f78a57a79e2c614191" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:51.881362 18805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:51.884617 18810 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c27799f097aa48f78a57a79e2c614191" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:51.885092 18810 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:51.885599 18326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Refusing update from remote peer c27799f097aa48f78a57a79e2c614191: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20260811 02:57:51.886564 18810 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260811 02:57:51.889710 18566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c27799f097aa48f78a57a79e2c614191. Latest consensus state: current_term: 3 leader_uuid: "c27799f097aa48f78a57a79e2c614191" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:51.889992 18566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:51.890415 18747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c27799f097aa48f78a57a79e2c614191" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:51.890731 18747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:51.970013 18654 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
17fb94eb8b7b4caea6f9669c114504f4 | 127.13.94.190:41293 | HEALTHY
91d3a9f0f5474fb9b0409772125153e2 | 127.13.94.189:40361 | HEALTHY
c27799f097aa48f78a57a79e2c614191 | 127.13.94.188:34185 | HEALTHY
All reported replicas are:
A = 17fb94eb8b7b4caea6f9669c114504f4
B = 91d3a9f0f5474fb9b0409772125153e2
C = c27799f097aa48f78a57a79e2c614191
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
A | A B C | 3 | 9 | Yes
B | A B C | 3 | 9 | Yes
C | A B C* | 3 | 9 | Yes
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 3 server(s) checked
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_reuseport | true | experimental | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb | hidden | 127.13.94.190:41293
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb | hidden | 127.13.94.189:40361
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb | hidden | 127.13.94.188:34185
tsk_num_rsa_bits | 512 | experimental | all 3 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.13.94.148:43463 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
e9aa9c71f0b44978856df6dbdc070eb9 | 127.13.94.129:35351 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.13.94.129 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.13.94.148:43463 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 3
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: master consensus error: there are master consensus conflicts
FAILED
I20260811 02:57:52.002197 13690 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 02:57:52.374492 18654 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
17fb94eb8b7b4caea6f9669c114504f4 | 127.13.94.190:41293 | HEALTHY
91d3a9f0f5474fb9b0409772125153e2 | 127.13.94.189:40361 | HEALTHY
c27799f097aa48f78a57a79e2c614191 | 127.13.94.188:34185 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 3 server(s) checked
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_reuseport | true | experimental | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb | hidden | 127.13.94.190:41293
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb | hidden | 127.13.94.189:40361
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb | hidden | 127.13.94.188:34185
tsk_num_rsa_bits | 512 | experimental | all 3 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.13.94.148:43463 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
e9aa9c71f0b44978856df6dbdc070eb9 | 127.13.94.129:35351 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.13.94.129 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.13.94.148:43463 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 3
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 02:57:52.406256 13690 log_verifier.cc:126] Checking tablet bd12ba4ba8ea4deebb7b059546592ce4
I20260811 02:57:52.422155 13690 log_verifier.cc:177] Verified matching terms for 2 ops in tablet bd12ba4ba8ea4deebb7b059546592ce4
I20260811 02:57:52.422434 13690 dynamic_multi_master-test.cc:688] Verifying the first table
I20260811 02:57:52.450268 18810 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c27799f097aa48f78a57a79e2c614191" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:57:52.450587 18810 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:52.473353 13690 dynamic_multi_master-test.cc:691] Creating and verifying the second table
I20260811 02:57:52.501802 18502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54924:
name: "second_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:52.553128 18654 tablet_service.cc:1467] Processing CreateTablet for tablet 101b9f8464f743639aa60f0c2d40517c (DEFAULT_TABLE table=second_table [id=8513df49c114424cb71e292126ac7eef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:52.554121 18654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 101b9f8464f743639aa60f0c2d40517c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:52.566176 18887 tablet_bootstrap.cc:492] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9: Bootstrap starting.
I20260811 02:57:52.568965 18887 tablet_bootstrap.cc:654] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:52.575827 18887 tablet_bootstrap.cc:492] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9: No bootstrap required, opened a new log
I20260811 02:57:52.576093 18887 ts_tablet_manager.cc:1397] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260811 02:57:52.577549 18887 raft_consensus.cc:348] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.577924 18887 raft_consensus.cc:374] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:52.578120 18887 raft_consensus.cc:729] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Initialized, Role: FOLLOWER
I20260811 02:57:52.578488 18887 consensus_queue.cc:260] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.578774 18887 raft_consensus.cc:388] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:52.578876 18887 raft_consensus.cc:482] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:52.579033 18887 raft_consensus.cc:3037] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:52.582937 18887 raft_consensus.cc:504] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.583333 18887 leader_election.cc:302] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9aa9c71f0b44978856df6dbdc070eb9; no voters:
I20260811 02:57:52.584020 18887 leader_election.cc:288] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:52.584352 18889 raft_consensus.cc:2781] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:52.585176 18726 heartbeater.cc:503] Master 127.13.94.188:34185 was elected leader, sending a full tablet report...
I20260811 02:57:52.586040 18887 ts_tablet_manager.cc:1428] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 02:57:52.589934 18889 raft_consensus.cc:686] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 LEADER]: Becoming Leader. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Running, Role: LEADER
I20260811 02:57:52.590355 18889 consensus_queue.cc:237] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.595186 18502 catalog_manager.cc:5495] T 101b9f8464f743639aa60f0c2d40517c P e9aa9c71f0b44978856df6dbdc070eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9aa9c71f0b44978856df6dbdc070eb9 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:52.660702 13690 dynamic_multi_master-test.cc:700] Pausing and resuming individual masters
I20260811 02:57:52.660800 13690 dynamic_multi_master-test.cc:704] Pausing master 17fb94eb8b7b4caea6f9669c114504f4, 127.13.94.190:41293
I20260811 02:57:52.737890 18502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54950:
name: "table-0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:52.751008 18654 tablet_service.cc:1467] Processing CreateTablet for tablet 7413faae4a5d4e46a5eaf9134590d8ad (DEFAULT_TABLE table=table-0 [id=1e6325c127ac48a0874ee010db309ed2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:52.751657 18654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7413faae4a5d4e46a5eaf9134590d8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:52.758881 18887 tablet_bootstrap.cc:492] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9: Bootstrap starting.
I20260811 02:57:52.760910 18887 tablet_bootstrap.cc:654] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:52.763288 18887 tablet_bootstrap.cc:492] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9: No bootstrap required, opened a new log
I20260811 02:57:52.763512 18887 ts_tablet_manager.cc:1397] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:57:52.764362 18887 raft_consensus.cc:348] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.764669 18887 raft_consensus.cc:374] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:52.764794 18887 raft_consensus.cc:729] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Initialized, Role: FOLLOWER
I20260811 02:57:52.765094 18887 consensus_queue.cc:260] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.765419 18887 raft_consensus.cc:388] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:52.765565 18887 raft_consensus.cc:482] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:52.765731 18887 raft_consensus.cc:3037] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:52.768026 18887 raft_consensus.cc:504] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.768326 18887 leader_election.cc:302] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9aa9c71f0b44978856df6dbdc070eb9; no voters:
I20260811 02:57:52.768580 18887 leader_election.cc:288] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:52.768641 18889 raft_consensus.cc:2781] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:52.768855 18889 raft_consensus.cc:686] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 LEADER]: Becoming Leader. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Running, Role: LEADER
I20260811 02:57:52.769115 18889 consensus_queue.cc:237] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.769866 18887 ts_tablet_manager.cc:1428] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:57:52.773000 18502 catalog_manager.cc:5495] T 7413faae4a5d4e46a5eaf9134590d8ad P e9aa9c71f0b44978856df6dbdc070eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9aa9c71f0b44978856df6dbdc070eb9 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:52.786149 13690 dynamic_multi_master-test.cc:704] Pausing master 91d3a9f0f5474fb9b0409772125153e2, 127.13.94.189:40361
I20260811 02:57:52.907279 18503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54964:
name: "table-1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:52.919795 18654 tablet_service.cc:1467] Processing CreateTablet for tablet fef3bcf325ae47318689d35f068e4fec (DEFAULT_TABLE table=table-1 [id=229f122d4ff04eb5b5007c9e07e78aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:52.920387 18654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fef3bcf325ae47318689d35f068e4fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:52.927780 18887 tablet_bootstrap.cc:492] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9: Bootstrap starting.
I20260811 02:57:52.929404 18887 tablet_bootstrap.cc:654] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:52.931891 18887 tablet_bootstrap.cc:492] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9: No bootstrap required, opened a new log
I20260811 02:57:52.932124 18887 ts_tablet_manager.cc:1397] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:57:52.932880 18887 raft_consensus.cc:348] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.933092 18887 raft_consensus.cc:374] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:52.933295 18887 raft_consensus.cc:729] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Initialized, Role: FOLLOWER
I20260811 02:57:52.933588 18887 consensus_queue.cc:260] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.933863 18887 raft_consensus.cc:388] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:52.934006 18887 raft_consensus.cc:482] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:52.934177 18887 raft_consensus.cc:3037] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:52.936820 18887 raft_consensus.cc:504] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.937176 18887 leader_election.cc:302] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9aa9c71f0b44978856df6dbdc070eb9; no voters:
I20260811 02:57:52.937453 18887 leader_election.cc:288] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:52.937609 18889 raft_consensus.cc:2781] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:52.937834 18889 raft_consensus.cc:686] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 LEADER]: Becoming Leader. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Running, Role: LEADER
I20260811 02:57:52.938017 18887 ts_tablet_manager.cc:1428] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:57:52.938122 18889 consensus_queue.cc:237] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:57:52.941037 18502 catalog_manager.cc:5495] T fef3bcf325ae47318689d35f068e4fec P e9aa9c71f0b44978856df6dbdc070eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9aa9c71f0b44978856df6dbdc070eb9 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:52.948787 13690 dynamic_multi_master-test.cc:704] Pausing master c27799f097aa48f78a57a79e2c614191, 127.13.94.188:34185
I20260811 02:57:54.447788 18948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c27799f097aa48f78a57a79e2c614191)
I20260811 02:57:54.448056 18948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:54.448997 18948 leader_election.cc:288] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91d3a9f0f5474fb9b0409772125153e2 (127.13.94.189:40361), c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185)
I20260811 02:57:54.449576 18420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "91d3a9f0f5474fb9b0409772125153e2" is_pre_election: true
I20260811 02:57:54.454550 18949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c27799f097aa48f78a57a79e2c614191)
I20260811 02:57:54.454784 18949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:54.457051 18949 leader_election.cc:288] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 17fb94eb8b7b4caea6f9669c114504f4 (127.13.94.190:41293), c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185)
I20260811 02:57:54.464738 18325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "91d3a9f0f5474fb9b0409772125153e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "17fb94eb8b7b4caea6f9669c114504f4" is_pre_election: true
I20260811 02:57:54.465199 18325 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91d3a9f0f5474fb9b0409772125153e2 for term 4 because replica has last-logged OpId of term: 3 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 19.
W20260811 02:57:56.211495 18946 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:52.992621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.94.188:34185 (local address 127.0.0.1:54978)
0811 02:57:53.003392 (+ 10771us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.003426 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.003599 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:56.210054 (+3206455us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.13.94.188:34185: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":10584,"thread_start_us":10418,"threads_started":1}
W20260811 02:57:56.412739 18297 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.94.188:34185 timed out after 1.962s (SENT)
W20260811 02:57:56.515450 18394 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.13.94.188:34185 for RPC RequestConsensusVote timed out after 2.057s (ON_OUTBOUND_QUEUE)
W20260811 02:57:57.490983 18950 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:54.457542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.94.188:34185 (local address 127.0.0.1:54994)
0811 02:57:54.459056 (+ 1514us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.459079 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.459194 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:57.489961 (+3030767us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.13.94.188:34185: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":1398,"mutex_wait_us":29,"thread_start_us":307,"threads_started":1}
W20260811 02:57:57.491302 18394 leader_election.cc:334] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185): Timed out: Client connection negotiation failed: client connection to 127.13.94.188:34185: received 0 of 4 requested bytes
I20260811 02:57:57.491462 18394 leader_election.cc:302] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91d3a9f0f5474fb9b0409772125153e2; no voters: 17fb94eb8b7b4caea6f9669c114504f4, c27799f097aa48f78a57a79e2c614191
I20260811 02:57:57.492168 18956 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260811 02:57:58.377418 18297 leader_election.cc:334] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185): Timed out: RequestConsensusVote RPC to 127.13.94.188:34185 timed out after 1.962s (SENT)
I20260811 02:57:58.377725 18297 leader_election.cc:302] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17fb94eb8b7b4caea6f9669c114504f4; no voters: 91d3a9f0f5474fb9b0409772125153e2, c27799f097aa48f78a57a79e2c614191
I20260811 02:57:58.378509 18957 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260811 02:57:59.282203 18946 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:56.215876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.94.188:34185 (local address 127.0.0.1:34906)
0811 02:57:56.216158 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.216180 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:56.216380 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:59.281999 (+3065619us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.13.94.188:34185: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 02:57:59.703893 18958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c27799f097aa48f78a57a79e2c614191)
I20260811 02:57:59.704151 18958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:57:59.704895 18958 leader_election.cc:288] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 17fb94eb8b7b4caea6f9669c114504f4 (127.13.94.190:41293), c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185)
I20260811 02:57:59.705470 18325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "91d3a9f0f5474fb9b0409772125153e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "17fb94eb8b7b4caea6f9669c114504f4" is_pre_election: true
I20260811 02:57:59.705900 18325 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91d3a9f0f5474fb9b0409772125153e2 for term 4 because replica has last-logged OpId of term: 3 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 19.
I20260811 02:58:00.273092 18960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c27799f097aa48f78a57a79e2c614191)
I20260811 02:58:00.273362 18960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:58:00.274214 18960 leader_election.cc:288] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91d3a9f0f5474fb9b0409772125153e2 (127.13.94.189:40361), c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185)
I20260811 02:58:00.274809 18420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "91d3a9f0f5474fb9b0409772125153e2" is_pre_election: true
I20260811 02:58:00.275048 18420 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fb94eb8b7b4caea6f9669c114504f4 in term 3.
I20260811 02:58:00.275625 18299 leader_election.cc:302] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fb94eb8b7b4caea6f9669c114504f4, 91d3a9f0f5474fb9b0409772125153e2; no voters:
I20260811 02:58:00.275970 18960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 02:58:00.276119 18960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c27799f097aa48f78a57a79e2c614191)
I20260811 02:58:00.276243 18960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:58:00.278456 18960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:58:00.279117 18960 leader_election.cc:288] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 4 election: Requested vote from peers 91d3a9f0f5474fb9b0409772125153e2 (127.13.94.189:40361), c27799f097aa48f78a57a79e2c614191 (127.13.94.188:34185)
I20260811 02:58:00.279704 18420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "91d3a9f0f5474fb9b0409772125153e2"
I20260811 02:58:00.279978 18420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:58:00.282114 18420 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fb94eb8b7b4caea6f9669c114504f4 in term 4.
I20260811 02:58:00.282631 18299 leader_election.cc:302] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fb94eb8b7b4caea6f9669c114504f4, 91d3a9f0f5474fb9b0409772125153e2; no voters:
I20260811 02:58:00.282994 18960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 02:58:00.283497 18960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [term 4 LEADER]: Becoming Leader. State: Replica: 17fb94eb8b7b4caea6f9669c114504f4, State: Running, Role: LEADER
I20260811 02:58:00.283875 18960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 3.21, Last appended by leader: 21, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:58:00.285341 18961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17fb94eb8b7b4caea6f9669c114504f4. Latest consensus state: current_term: 4 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:58:00.285812 18961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:00.286402 18963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:00.288074 18963 catalog_manager.cc:669] Loaded metadata for table table-0 [id=1e6325c127ac48a0874ee010db309ed2]
I20260811 02:58:00.288501 18963 catalog_manager.cc:669] Loaded metadata for table table-1 [id=229f122d4ff04eb5b5007c9e07e78aac]
I20260811 02:58:00.288828 18963 catalog_manager.cc:669] Loaded metadata for table first_table [id=7fe9c6d5429d444d82a87d5f32fa61ea]
I20260811 02:58:00.289155 18963 catalog_manager.cc:669] Loaded metadata for table second_table [id=8513df49c114424cb71e292126ac7eef]
I20260811 02:58:00.290499 18963 tablet_loader.cc:96] loaded metadata for tablet 101b9f8464f743639aa60f0c2d40517c (table second_table [id=8513df49c114424cb71e292126ac7eef])
I20260811 02:58:00.290872 18963 tablet_loader.cc:96] loaded metadata for tablet 7413faae4a5d4e46a5eaf9134590d8ad (table table-0 [id=1e6325c127ac48a0874ee010db309ed2])
I20260811 02:58:00.291172 18963 tablet_loader.cc:96] loaded metadata for tablet bd12ba4ba8ea4deebb7b059546592ce4 (table first_table [id=7fe9c6d5429d444d82a87d5f32fa61ea])
I20260811 02:58:00.291476 18963 tablet_loader.cc:96] loaded metadata for tablet fef3bcf325ae47318689d35f068e4fec (table table-1 [id=229f122d4ff04eb5b5007c9e07e78aac])
I20260811 02:58:00.291810 18963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:58:00.292742 18963 catalog_manager.cc:1259] Loaded cluster ID: 7c9307eb863047528c0daea25d8f82d8
I20260811 02:58:00.292846 18963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:00.294195 18963 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:00.295071 18963 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4: Loaded TSK: 0
I20260811 02:58:00.295596 18963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:58:00.714986 18420 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 4 FOLLOWER]: Refusing update from remote peer 17fb94eb8b7b4caea6f9669c114504f4: Log matching property violated. Preceding OpId in replica: term: 3 index: 19. Preceding OpId from leader: term: 4 index: 22. (index mismatch)
I20260811 02:58:00.715900 18961 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 18, Time since last communication: 0.000s
I20260811 02:58:00.719826 18964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17fb94eb8b7b4caea6f9669c114504f4. Latest consensus state: current_term: 4 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:58:00.720230 18964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:00.722033 18960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:58:00.722397 18960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:00.722421 18964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:58:00.722661 18964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:01.966804 18727 heartbeater.cc:503] Master 127.13.94.190:41293 was elected leader, sending a full tablet report...
W20260811 02:58:02.354265 18946 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:59.289497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.94.188:34185 (local address 127.0.0.1:34916)
0811 02:57:59.289800 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:59.289824 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:59.289974 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:02.354019 (+3064045us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.13.94.188:34185: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 02:58:02.418666 18314 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47772:
name: "table-2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:02.443089 18654 tablet_service.cc:1467] Processing CreateTablet for tablet d7def735f0d84fb59674121230b8d98b (DEFAULT_TABLE table=table-2 [id=56893d5365f940528f65cfb25f3a9587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:02.443610 18654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7def735f0d84fb59674121230b8d98b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:02.448874 18996 tablet_bootstrap.cc:492] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9: Bootstrap starting.
I20260811 02:58:02.450515 18996 tablet_bootstrap.cc:654] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:02.452986 18996 tablet_bootstrap.cc:492] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9: No bootstrap required, opened a new log
I20260811 02:58:02.453142 18996 ts_tablet_manager.cc:1397] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:02.453953 18996 raft_consensus.cc:348] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:58:02.454216 18996 raft_consensus.cc:374] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:02.454337 18996 raft_consensus.cc:729] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Initialized, Role: FOLLOWER
I20260811 02:58:02.454576 18996 consensus_queue.cc:260] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:58:02.454768 18996 raft_consensus.cc:388] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:02.454854 18996 raft_consensus.cc:482] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:02.455003 18996 raft_consensus.cc:3037] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:02.456838 18996 raft_consensus.cc:504] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:58:02.457110 18996 leader_election.cc:302] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9aa9c71f0b44978856df6dbdc070eb9; no voters:
I20260811 02:58:02.457607 18996 leader_election.cc:288] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:02.457824 18999 raft_consensus.cc:2781] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:02.458504 18996 ts_tablet_manager.cc:1428] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260811 02:58:02.458454 18999 raft_consensus.cc:686] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [term 1 LEADER]: Becoming Leader. State: Replica: e9aa9c71f0b44978856df6dbdc070eb9, State: Running, Role: LEADER
I20260811 02:58:02.458853 18999 consensus_queue.cc:237] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } }
I20260811 02:58:02.461473 18314 catalog_manager.cc:5495] T d7def735f0d84fb59674121230b8d98b P e9aa9c71f0b44978856df6dbdc070eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9aa9c71f0b44978856df6dbdc070eb9 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9aa9c71f0b44978856df6dbdc070eb9" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:02.468837 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 18571
I20260811 02:58:02.471791 18502 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.13.94.129:53039 (request call id 9) took 9523 ms. Trace:
I20260811 02:58:02.471930 18502 rpcz_store.cc:276] 0811 02:57:52.948330 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 02:57:52.948382 (+ 52us) service_pool.cc:227] Handling call
0811 02:58:02.471723 (+9523341us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260811 02:58:02.473150 18509 raft_consensus.cc:3032] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 3 LEADER]: Stepping down as leader of term 3
I20260811 02:58:02.473253 18509 raft_consensus.cc:729] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c27799f097aa48f78a57a79e2c614191, State: Running, Role: LEADER
I20260811 02:58:02.473646 18509 consensus_queue.cc:260] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 3.21, Last appended by leader: 21, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } }
I20260811 02:58:02.474488 18509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 3 FOLLOWER]: Advancing to term 4
I20260811 02:58:02.474692 18515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "c27799f097aa48f78a57a79e2c614191" is_pre_election: true
I20260811 02:58:02.475387 18513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "c27799f097aa48f78a57a79e2c614191" is_pre_election: true
I20260811 02:58:02.476104 18511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "c27799f097aa48f78a57a79e2c614191" is_pre_election: true
I20260811 02:58:02.476734 18510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17fb94eb8b7b4caea6f9669c114504f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "c27799f097aa48f78a57a79e2c614191"
I20260811 02:58:02.502173 18509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [term 4 FOLLOWER]: Refusing update from remote peer 17fb94eb8b7b4caea6f9669c114504f4: Log matching property violated. Preceding OpId in replica: term: 3 index: 21. Preceding OpId from leader: term: 4 index: 22. (index mismatch)
W20260811 02:58:02.503680 18759 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:02.477794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54994 (local address 127.13.94.188:34185)
0811 02:58:02.502828 (+ 25034us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.502834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.502854 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:58:02.502929 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.502937 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.503031 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.503280 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54994: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":24922,"threads_started":1}
W20260811 02:58:02.505769 18759 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:02.483563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54978 (local address 127.13.94.188:34185)
0811 02:58:02.504937 (+ 21374us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.504943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.504962 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:58:02.505006 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.505011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.505075 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.505272 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54978: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":21271,"thread_start_us":17009,"threads_started":1}
W20260811 02:58:02.507766 18759 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:02.487930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34906 (local address 127.13.94.188:34185)
0811 02:58:02.507059 (+ 19129us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.507067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.507091 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:58:02.507147 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.507151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.507233 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.507459 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34906: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":18999,"threads_started":1}
W20260811 02:58:02.513530 19008 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:02.512189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34916 (local address 127.13.94.188:34185)
0811 02:58:02.512760 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.512766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.512789 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:58:02.512837 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.512841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.512922 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.513221 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34916: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":433}
I20260811 02:58:02.513327 18973 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 17fb94eb8b7b4caea6f9669c114504f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260811 02:58:02.532428 18868 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17fb94eb8b7b4caea6f9669c114504f4. Latest consensus state: current_term: 4 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:58:02.532910 18868 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:02.532626 18510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "91d3a9f0f5474fb9b0409772125153e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "c27799f097aa48f78a57a79e2c614191" is_pre_election: true
I20260811 02:58:02.533515 18810 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "17fb94eb8b7b4caea6f9669c114504f4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "17fb94eb8b7b4caea6f9669c114504f4" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 41293 } } peers { permanent_uuid: "91d3a9f0f5474fb9b0409772125153e2" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 40361 } } peers { permanent_uuid: "c27799f097aa48f78a57a79e2c614191" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 34185 } attrs { promote: false } } }
I20260811 02:58:02.533838 18810 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27799f097aa48f78a57a79e2c614191 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:02.533839 18394 leader_election.cc:302] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91d3a9f0f5474fb9b0409772125153e2; no voters: 17fb94eb8b7b4caea6f9669c114504f4, c27799f097aa48f78a57a79e2c614191
I20260811 02:58:02.547247 19013 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 91d3a9f0f5474fb9b0409772125153e2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 02:58:02.550356 19010 mvcc.cc:205] Tried to move back new op lower bound from 7317164369767444480 to 7317164360845180928. Current Snapshot: MvccSnapshot[applied={T|T < 7317164369767444480 or (T in {7317164369767444480})}]
I20260811 02:58:02.555164 18720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:58:02.706737 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 18571
I20260811 02:58:02.747666 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 18281
I20260811 02:58:02.749073 18353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:58:02.883037 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 18281
I20260811 02:58:02.913025 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 18373
I20260811 02:58:02.914280 18440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:58:03.039047 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 18373
I20260811 02:58:03.066424 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 18451
I20260811 02:58:03.067574 18534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:58:03.195544 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 18451
2026-08-11T02:58:03Z chronyd exiting
[ OK ] ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL/3 (24564 ms)
[----------] 1 test from ParameterizedAddMasterTest (24564 ms total)
[----------] 1 test from ParameterizedRemoveMasterTest
[ RUN ] ParameterizedRemoveMasterTest.TestRemoveMaster/1
2026-08-11T02:58:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:58:03Z Disabled control of system clock
I20260811 02:58:03.290282 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:46511
--webserver_interface=127.13.94.190
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:46511,127.13.94.189:37795
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
W20260811 02:58:03.658209 19023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:03.658702 19023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:03.658874 19023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:03.667167 19023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:03.667315 19023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:03.667407 19023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:03.667513 19023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:58:03.677172 19023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:46511,127.13.94.189:37795
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:46511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:58:03.677654 19023 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:46511,127.13.94.189:37795'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:58:03.678851 19023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:03.680850 19023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:03.694680 19029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:03.694759 19031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:03.694895 19023 server_base.cc:1034] running on GCE node
W20260811 02:58:03.694759 19028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:03.696282 19023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:03.698017 19023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:03.699265 19023 hybrid_clock.cc:648] HybridClock initialized: now 1786417083699211 us; error 47 us; skew 500 ppm
I20260811 02:58:03.699671 19023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:58:03.702332 19023 webserver.cc:460] Webserver started at http://127.13.94.190:34111/ using document root <none> and password file <none>
I20260811 02:58:03.703135 19023 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:03.703276 19023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:03.703626 19023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:03.706565 19023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/instance:
uuid: "92c330161dda44689dcfbde0de7f2fcf"
format_stamp: "Formatted at 2026-08-11 02:58:03 on dist-test-slave-p3t5"
I20260811 02:58:03.707456 19023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal/instance:
uuid: "92c330161dda44689dcfbde0de7f2fcf"
format_stamp: "Formatted at 2026-08-11 02:58:03 on dist-test-slave-p3t5"
I20260811 02:58:03.713446 19023 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 02:58:03.718146 19038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:03.719774 19023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:58:03.720029 19023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "92c330161dda44689dcfbde0de7f2fcf"
format_stamp: "Formatted at 2026-08-11 02:58:03 on dist-test-slave-p3t5"
I20260811 02:58:03.720297 19023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:03.736531 19023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:03.737519 19023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:03.737897 19023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:03.758414 19023 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:46511
I20260811 02:58:03.758451 19091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:46511 every 8 connection(s)
I20260811 02:58:03.760094 19023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:58:03.764348 19092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:03.769201 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 19023
I20260811 02:58:03.769428 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal/instance
I20260811 02:58:03.770690 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:37795
--webserver_interface=127.13.94.189
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:46511,127.13.94.189:37795
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
I20260811 02:58:03.770970 19092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:58:03.790766 19092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } has no permanent_uuid. Determining permanent_uuid...
W20260811 02:58:03.792368 19042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111)
W20260811 02:58:03.794322 19092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:37795: Network error: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111)
I20260811 02:58:03.843827 19092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } attempt: 1
W20260811 02:58:03.846190 19092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:37795: Network error: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111)
I20260811 02:58:03.914546 19092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } attempt: 2
W20260811 02:58:03.916922 19092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:37795: Network error: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111)
I20260811 02:58:04.008339 19092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } attempt: 3
W20260811 02:58:04.011081 19092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:37795: Network error: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111)
W20260811 02:58:04.140961 19096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:04.141299 19096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:04.141379 19096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:04.148919 19096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:04.149046 19096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:04.149089 19096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:04.149142 19096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:58:04.154395 19092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } attempt: 4
W20260811 02:58:04.156677 19092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:37795: Network error: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111)
I20260811 02:58:04.159188 19096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:46511,127.13.94.189:37795
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:37795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:58:04.159575 19096 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:46511,127.13.94.189:37795'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:58:04.160720 19096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:04.162664 19096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:04.175953 19105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:04.176010 19104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:04.176048 19108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:04.176769 19096 server_base.cc:1034] running on GCE node
I20260811 02:58:04.177767 19096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:04.179370 19096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:04.180632 19096 hybrid_clock.cc:648] HybridClock initialized: now 1786417084180584 us; error 51 us; skew 500 ppm
I20260811 02:58:04.181038 19096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:58:04.183971 19096 webserver.cc:460] Webserver started at http://127.13.94.189:33759/ using document root <none> and password file <none>
I20260811 02:58:04.184832 19096 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:04.184984 19096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:04.185364 19096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:04.188555 19096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/data/instance:
uuid: "77a988f1e82d40f3a831be7988cf8426"
format_stamp: "Formatted at 2026-08-11 02:58:04 on dist-test-slave-p3t5"
I20260811 02:58:04.189597 19096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/wal/instance:
uuid: "77a988f1e82d40f3a831be7988cf8426"
format_stamp: "Formatted at 2026-08-11 02:58:04 on dist-test-slave-p3t5"
I20260811 02:58:04.195940 19096 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 02:58:04.199966 19114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:04.201567 19096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:58:04.201848 19096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "77a988f1e82d40f3a831be7988cf8426"
format_stamp: "Formatted at 2026-08-11 02:58:04 on dist-test-slave-p3t5"
I20260811 02:58:04.202117 19096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:04.228529 19096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:04.229506 19096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:04.229929 19096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:04.249437 19096 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:37795
I20260811 02:58:04.249493 19168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:37795 every 8 connection(s)
I20260811 02:58:04.251000 19096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:58:04.254838 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 19096
I20260811 02:58:04.255034 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-1/wal/instance
I20260811 02:58:04.255553 19169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:04.264973 19169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:58:04.280045 19169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:58:04.291157 19169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426: Bootstrap starting.
I20260811 02:58:04.294780 19169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:04.295904 19169 log.cc:824] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:04.298817 19169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426: No bootstrap required, opened a new log
I20260811 02:58:04.306605 19169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } }
I20260811 02:58:04.306972 19169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:04.307097 19169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a988f1e82d40f3a831be7988cf8426, State: Initialized, Role: FOLLOWER
I20260811 02:58:04.307654 19169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } }
I20260811 02:58:04.308490 19175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:04.308830 19175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:04.309556 19169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:58:04.319682 19187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:58:04.319892 19187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:58:04.412468 19175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:58:04.412868 19175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } }
I20260811 02:58:04.414448 19175 leader_election.cc:288] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92c330161dda44689dcfbde0de7f2fcf (127.13.94.190:46511)
I20260811 02:58:04.455916 19092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } attempt: 5
I20260811 02:58:04.471052 19092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: Bootstrap starting.
I20260811 02:58:04.474697 19092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:04.476192 19092 log.cc:824] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:04.479875 19092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: No bootstrap required, opened a new log
W20260811 02:58:04.482143 19120 tablet.cc:2369] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:58:04.491273 19092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } }
I20260811 02:58:04.491678 19092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:04.491788 19092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92c330161dda44689dcfbde0de7f2fcf, State: Initialized, Role: FOLLOWER
I20260811 02:58:04.492528 19092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } }
I20260811 02:58:04.493640 19190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:04.494103 19190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:04.494942 19092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:58:04.495043 19066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "77a988f1e82d40f3a831be7988cf8426" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92c330161dda44689dcfbde0de7f2fcf" is_pre_election: true
I20260811 02:58:04.495743 19066 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77a988f1e82d40f3a831be7988cf8426 in term 0.
I20260811 02:58:04.496763 19117 leader_election.cc:302] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 77a988f1e82d40f3a831be7988cf8426, 92c330161dda44689dcfbde0de7f2fcf; no voters:
I20260811 02:58:04.497331 19175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:58:04.497517 19175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:58:04.497663 19175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:04.500349 19175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } }
I20260811 02:58:04.501039 19175 leader_election.cc:288] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [CANDIDATE]: Term 1 election: Requested vote from peers 92c330161dda44689dcfbde0de7f2fcf (127.13.94.190:46511)
I20260811 02:58:04.502244 19066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "77a988f1e82d40f3a831be7988cf8426" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92c330161dda44689dcfbde0de7f2fcf"
I20260811 02:58:04.502509 19066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:04.504827 19066 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77a988f1e82d40f3a831be7988cf8426 in term 1.
I20260811 02:58:04.505270 19117 leader_election.cc:302] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 77a988f1e82d40f3a831be7988cf8426, 92c330161dda44689dcfbde0de7f2fcf; no voters:
I20260811 02:58:04.505654 19175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:04.506346 19175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 1 LEADER]: Becoming Leader. State: Replica: 77a988f1e82d40f3a831be7988cf8426, State: Running, Role: LEADER
I20260811 02:58:04.507057 19175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } }
I20260811 02:58:04.509989 19197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77a988f1e82d40f3a831be7988cf8426. Latest consensus state: current_term: 1 leader_uuid: "77a988f1e82d40f3a831be7988cf8426" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:04.510483 19197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:04.511538 19204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:04.514325 19204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 02:58:04.520293 19205 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:58:04.520713 19205 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:58:04.527295 19066 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 1 FOLLOWER]: Refusing update from remote peer 77a988f1e82d40f3a831be7988cf8426: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:04.528324 19197 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:04.539002 19190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77a988f1e82d40f3a831be7988cf8426. Latest consensus state: current_term: 1 leader_uuid: "77a988f1e82d40f3a831be7988cf8426" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:04.539304 19190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:04.542841 19207 mvcc.cc:205] Tried to move back new op lower bound from 7317164378210471936 to 7317164378145021952. Current Snapshot: MvccSnapshot[applied={T|T < 7317164378210471936}]
I20260811 02:58:04.568585 19197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "77a988f1e82d40f3a831be7988cf8426" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:04.569037 19197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:04.572114 19190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "77a988f1e82d40f3a831be7988cf8426" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:04.572417 19190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:04.573935 19204 catalog_manager.cc:1347] Generated new cluster ID: 5a7ed646581f489888943360f34589f4
I20260811 02:58:04.574077 19204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:04.595247 19204 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:58:04.597016 19204 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:04.615233 19204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426: Generated new TSK 0
I20260811 02:58:04.616286 19204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:58:04.627380 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:0
--local_ip_for_outbound_sockets=127.13.94.129
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:46511,127.13.94.189:37795
--builtin_ntp_servers=127.13.94.148:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260811 02:58:05.013206 19212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:05.013656 19212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:05.013833 19212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:05.022228 19212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:05.022466 19212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:58:05.032375 19212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:46511,127.13.94.189:37795
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:58:05.034036 19212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:05.036118 19212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:05.050843 19220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:05.050772 19217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:05.050810 19218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:05.052078 19212 server_base.cc:1034] running on GCE node
I20260811 02:58:05.052773 19212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:05.053851 19212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:05.055087 19212 hybrid_clock.cc:648] HybridClock initialized: now 1786417085055016 us; error 36 us; skew 500 ppm
I20260811 02:58:05.055398 19212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:58:05.057989 19212 webserver.cc:460] Webserver started at http://127.13.94.129:42735/ using document root <none> and password file <none>
I20260811 02:58:05.058763 19212 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:05.058919 19212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:05.059247 19212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:05.062157 19212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/instance:
uuid: "62d5f705c166431a943d00e961f51819"
format_stamp: "Formatted at 2026-08-11 02:58:05 on dist-test-slave-p3t5"
I20260811 02:58:05.062991 19212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance:
uuid: "62d5f705c166431a943d00e961f51819"
format_stamp: "Formatted at 2026-08-11 02:58:05 on dist-test-slave-p3t5"
I20260811 02:58:05.068689 19212 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 02:58:05.072636 19226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:05.074312 19212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 02:58:05.074515 19212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "62d5f705c166431a943d00e961f51819"
format_stamp: "Formatted at 2026-08-11 02:58:05 on dist-test-slave-p3t5"
I20260811 02:58:05.074731 19212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:05.087529 19212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:05.088423 19212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:05.088733 19212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:05.089818 19212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:05.091451 19212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:05.091558 19212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:05.091713 19212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:05.091789 19212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:05.128182 19212 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:36937
I20260811 02:58:05.128276 19344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:36937 every 8 connection(s)
I20260811 02:58:05.129590 19212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:58:05.137526 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 19212
I20260811 02:58:05.137840 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance
I20260811 02:58:05.147010 19345 heartbeater.cc:347] Connected to a master server at 127.13.94.189:37795
I20260811 02:58:05.147382 19345 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:05.148202 19345 heartbeater.cc:511] Master 127.13.94.189:37795 requested a full tablet report, sending...
I20260811 02:58:05.149008 19346 heartbeater.cc:347] Connected to a master server at 127.13.94.190:46511
I20260811 02:58:05.149160 19346 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:05.149590 19346 heartbeater.cc:511] Master 127.13.94.190:46511 requested a full tablet report, sending...
I20260811 02:58:05.151461 19132 ts_manager.cc:194] Registered new tserver with Master: 62d5f705c166431a943d00e961f51819 (127.13.94.129:36937)
I20260811 02:58:05.151598 19055 ts_manager.cc:194] Registered new tserver with Master: 62d5f705c166431a943d00e961f51819 (127.13.94.129:36937)
I20260811 02:58:05.153461 19132 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.94.129:49053
I20260811 02:58:05.159166 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:58:05.164217 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:58:05.173913 13690 dynamic_multi_master-test.cc:972] Transferring leadership to master: 92c330161dda44689dcfbde0de7f2fcf
I20260811 02:58:05.177037 19143 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "77a988f1e82d40f3a831be7988cf8426"
mode: GRACEFUL
new_leader_uuid: "92c330161dda44689dcfbde0de7f2fcf"
from {username='slave'} at 127.0.0.1:57340
I20260811 02:58:05.177424 19143 raft_consensus.cc:595] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 1 LEADER]: Received request to transfer leadership to TS 92c330161dda44689dcfbde0de7f2fcf
I20260811 02:58:05.523891 19205 catalog_manager.cc:1259] Loaded cluster ID: 5a7ed646581f489888943360f34589f4
I20260811 02:58:05.524031 19205 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: loading cluster ID for follower catalog manager: success
I20260811 02:58:05.528352 19205 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: acquiring CA information for follower catalog manager: success
I20260811 02:58:05.531116 19205 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:58:05.717134 19354 raft_consensus.cc:979] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426: : Instructing follower 92c330161dda44689dcfbde0de7f2fcf to start an election
I20260811 02:58:05.717307 19354 raft_consensus.cc:1067] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 1 LEADER]: Signalling peer 92c330161dda44689dcfbde0de7f2fcf to start an election
I20260811 02:58:05.718291 19066 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "92c330161dda44689dcfbde0de7f2fcf"
from {username='slave'} at 127.0.0.1:44094
I20260811 02:58:05.718564 19066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 02:58:05.718678 19066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:58:05.720733 19066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } }
I20260811 02:58:05.721854 19066 leader_election.cc:288] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [CANDIDATE]: Term 2 election: Requested vote from peers 77a988f1e82d40f3a831be7988cf8426 (127.13.94.189:37795)
I20260811 02:58:05.722625 19143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92c330161dda44689dcfbde0de7f2fcf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "77a988f1e82d40f3a831be7988cf8426"
I20260811 02:58:05.722880 19143 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 1 LEADER]: Stepping down as leader of term 1
I20260811 02:58:05.722968 19143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 77a988f1e82d40f3a831be7988cf8426, State: Running, Role: LEADER
I20260811 02:58:05.723264 19143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } }
I20260811 02:58:05.723737 19143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:58:05.725879 19143 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92c330161dda44689dcfbde0de7f2fcf in term 2.
I20260811 02:58:05.726426 19042 leader_election.cc:302] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 77a988f1e82d40f3a831be7988cf8426, 92c330161dda44689dcfbde0de7f2fcf; no voters:
I20260811 02:58:05.727272 19356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:58:05.727720 19356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 LEADER]: Becoming Leader. State: Replica: 92c330161dda44689dcfbde0de7f2fcf, State: Running, Role: LEADER
I20260811 02:58:05.728168 19356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } }
I20260811 02:58:05.729884 19357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92c330161dda44689dcfbde0de7f2fcf. Latest consensus state: current_term: 2 leader_uuid: "92c330161dda44689dcfbde0de7f2fcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:05.730185 19357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:05.730825 19359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:05.732532 19359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:58:05.733358 19359 catalog_manager.cc:1259] Loaded cluster ID: 5a7ed646581f489888943360f34589f4
I20260811 02:58:05.733471 19359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:05.734730 19359 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:05.735699 19359 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: Loaded TSK: 0
I20260811 02:58:05.736624 19359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:58:06.156783 19345 heartbeater.cc:503] Master 127.13.94.189:37795 was elected leader, sending a full tablet report...
I20260811 02:58:06.272008 19143 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [term 2 FOLLOWER]: Refusing update from remote peer 92c330161dda44689dcfbde0de7f2fcf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260811 02:58:06.273180 19368 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 02:58:06.275894 19369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92c330161dda44689dcfbde0de7f2fcf. Latest consensus state: current_term: 2 leader_uuid: "92c330161dda44689dcfbde0de7f2fcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:06.276204 19369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:06.279006 19368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92c330161dda44689dcfbde0de7f2fcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:06.279276 19368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:06.281700 19369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92c330161dda44689dcfbde0de7f2fcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:06.281985 19369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:06.285456 19055 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54988:
name: "first_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:06.321132 19276 tablet_service.cc:1467] Processing CreateTablet for tablet a1e3d5a281884d0394daa5308da86ddd (DEFAULT_TABLE table=first_table [id=07f1f8165f604852abe37e7bcaf72f34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:06.322810 19276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1e3d5a281884d0394daa5308da86ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:06.323179 19187 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 77a988f1e82d40f3a831be7988cf8426: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:58:06.334333 19379 tablet_bootstrap.cc:492] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Bootstrap starting.
I20260811 02:58:06.337997 19379 tablet_bootstrap.cc:654] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:06.339183 19379 log.cc:824] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:06.342335 19379 tablet_bootstrap.cc:492] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: No bootstrap required, opened a new log
I20260811 02:58:06.342661 19379 ts_tablet_manager.cc:1397] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260811 02:58:06.350953 19379 raft_consensus.cc:348] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d5f705c166431a943d00e961f51819" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 36937 } }
I20260811 02:58:06.351397 19379 raft_consensus.cc:374] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:06.351539 19379 raft_consensus.cc:729] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d5f705c166431a943d00e961f51819, State: Initialized, Role: FOLLOWER
I20260811 02:58:06.352201 19379 consensus_queue.cc:260] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d5f705c166431a943d00e961f51819" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 36937 } }
I20260811 02:58:06.352483 19379 raft_consensus.cc:388] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:06.352653 19379 raft_consensus.cc:482] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:06.352854 19379 raft_consensus.cc:3037] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:06.355367 19379 raft_consensus.cc:504] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d5f705c166431a943d00e961f51819" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 36937 } }
I20260811 02:58:06.356071 19379 leader_election.cc:302] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62d5f705c166431a943d00e961f51819; no voters:
I20260811 02:58:06.356784 19379 leader_election.cc:288] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:06.357013 19381 raft_consensus.cc:2781] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:06.357802 19381 raft_consensus.cc:686] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 1 LEADER]: Becoming Leader. State: Replica: 62d5f705c166431a943d00e961f51819, State: Running, Role: LEADER
I20260811 02:58:06.358079 19346 heartbeater.cc:503] Master 127.13.94.190:46511 was elected leader, sending a full tablet report...
I20260811 02:58:06.358461 19381 consensus_queue.cc:237] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d5f705c166431a943d00e961f51819" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 36937 } }
I20260811 02:58:06.359068 19379 ts_tablet_manager.cc:1428] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
I20260811 02:58:06.365392 19055 catalog_manager.cc:5495] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62d5f705c166431a943d00e961f51819 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "62d5f705c166431a943d00e961f51819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d5f705c166431a943d00e961f51819" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:06.376452 13690 dynamic_multi_master-test.cc:990] Shutting down the master to be removed
I20260811 02:58:06.376659 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 19096
I20260811 02:58:06.377805 19164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:58:06.486994 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 19096
I20260811 02:58:06.513110 13690 dynamic_multi_master-test.cc:992] Detecting transition to terminal FAILED state
W20260811 02:58:06.791749 19042 consensus_peers.cc:487] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf -> Peer 77a988f1e82d40f3a831be7988cf8426 (127.13.94.189:37795): Couldn't send request to peer 77a988f1e82d40f3a831be7988cf8426. Status: Network error: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 02:58:07.283437 19368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92c330161dda44689dcfbde0de7f2fcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:07.283811 19368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: This master's current role is: LEADER
W20260811 02:58:07.370069 19345 heartbeater.cc:644] Failed to heartbeat to 127.13.94.189:37795 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111)
W20260811 02:58:08.966305 19042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20260811 02:58:09.350029 19042 consensus_peers.cc:487] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf -> Peer 77a988f1e82d40f3a831be7988cf8426 (127.13.94.189:37795): Couldn't send request to peer 77a988f1e82d40f3a831be7988cf8426. Status: Network error: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 02:58:10.749394 19406 consensus_queue.cc:579] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [LEADER]: Leader has been unable to successfully communicate with peer 77a988f1e82d40f3a831be7988cf8426 for more than 4 seconds (4.378s)
I20260811 02:58:10.750227 19406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92c330161dda44689dcfbde0de7f2fcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } peers { permanent_uuid: "77a988f1e82d40f3a831be7988cf8426" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 37795 } } }
I20260811 02:58:10.750651 19406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: This master's current role is: LEADER
W20260811 02:58:11.898733 19042 consensus_peers.cc:487] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf -> Peer 77a988f1e82d40f3a831be7988cf8426 (127.13.94.189:37795): Couldn't send request to peer 77a988f1e82d40f3a831be7988cf8426. Status: Network error: Client connection negotiation failed: client connection to 127.13.94.189:37795: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260811 02:58:11.963135 19419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:11.963485 19419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:11.970835 19419 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20260811 02:58:12.011195 19055 catalog_manager.cc:6979] Initiating ChangeConfig request to remove master 127.13.94.189:37795: tablet_id: "00000000000000000000000000000000"
type: REMOVE_PEER
server {
permanent_uuid: "77a988f1e82d40f3a831be7988cf8426"
}
dest_uuid: "92c330161dda44689dcfbde0de7f2fcf"
cas_config_opid_index: -1
I20260811 02:58:12.012192 19055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } }
I20260811 02:58:12.013700 19420 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index -1 to 9, VOTER 77a988f1e82d40f3a831be7988cf8426 (127.13.94.189) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } }
I20260811 02:58:12.017081 19420 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to remove master 127.13.94.189:37795
I20260811 02:58:12.017706 19437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "92c330161dda44689dcfbde0de7f2fcf" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } }
I20260811 02:58:12.018005 19437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:12.018190 19419 tool_action_master.cc:566] Successfully removed master 127.13.94.189:37795 from the cluster. Please follow the next steps from the Kudu administration documentation on "Removing Kudu Masters from a Multi-Master Deployment" or "Recovering from a dead Kudu Master in a Multi-Master Deployment" as appropriate.
I20260811 02:58:12.201404 19276 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
92c330161dda44689dcfbde0de7f2fcf | 127.13.94.190:46511 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.13.94.148:46009 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
62d5f705c166431a943d00e961f51819 | 127.13.94.129:36937 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.13.94.129 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.13.94.148:46009 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 2 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 02:58:12.234758 13690 log_verifier.cc:126] Checking tablet a1e3d5a281884d0394daa5308da86ddd
I20260811 02:58:12.243482 13690 log_verifier.cc:177] Verified matching terms for 1 ops in tablet a1e3d5a281884d0394daa5308da86ddd
E20260811 02:58:12.683456 19054 master_service.cc:344] Failed removing master 127.13.94.189:37795. Not found: Master 127.13.94.189:37795 not found
I20260811 02:58:12.769161 13690 dynamic_multi_master-test.cc:1021] Transferring leadership to master: 77a988f1e82d40f3a831be7988cf8426
I20260811 02:58:12.773226 19066 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "92c330161dda44689dcfbde0de7f2fcf"
mode: GRACEFUL
new_leader_uuid: "77a988f1e82d40f3a831be7988cf8426"
from {username='slave'} at 127.0.0.1:44090
I20260811 02:58:12.773589 19066 raft_consensus.cc:595] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 LEADER]: Received request to transfer leadership to TS 77a988f1e82d40f3a831be7988cf8426
I20260811 02:58:12.773828 19066 raft_consensus.cc:619] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 LEADER]: Rejecting request to transfer leadership because tablet server 77a988f1e82d40f3a831be7988cf8426 is not a voter in the active config
I20260811 02:58:12.774963 13690 dynamic_multi_master-test.cc:1028] Shutting down the old cluster
I20260811 02:58:12.775152 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 19212
I20260811 02:58:12.783062 19340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:58:12.898088 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 19212
I20260811 02:58:12.924772 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 19023
I20260811 02:58:12.926141 19087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:58:13.040782 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 19023
2026-08-11T02:58:13Z chronyd exiting
I20260811 02:58:13.080907 13690 dynamic_multi_master-test.cc:1031] Bringing up the migrated cluster
2026-08-11T02:58:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:58:13Z Disabled control of system clock
I20260811 02:58:13.127643 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:46511
--webserver_interface=127.13.94.190
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:46511
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
W20260811 02:58:13.487946 19491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:13.488286 19491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:13.488364 19491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:13.495805 19491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:13.495924 19491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:13.495971 19491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:13.496021 19491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:58:13.505326 19491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:46511
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:46511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:58:13.506889 19491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:13.508710 19491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:13.521292 19496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:13.521313 19497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:13.521346 19499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:13.522377 19491 server_base.cc:1034] running on GCE node
I20260811 02:58:13.523407 19491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:13.524999 19491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:13.526285 19491 hybrid_clock.cc:648] HybridClock initialized: now 1786417093526194 us; error 58 us; skew 500 ppm
I20260811 02:58:13.526686 19491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:58:13.529307 19491 webserver.cc:460] Webserver started at http://127.13.94.190:44445/ using document root <none> and password file <none>
I20260811 02:58:13.530140 19491 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:13.530310 19491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:13.535969 19491 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:58:13.539147 19505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:13.540666 19491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:58:13.540885 19491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "92c330161dda44689dcfbde0de7f2fcf"
format_stamp: "Formatted at 2026-08-11 02:58:03 on dist-test-slave-p3t5"
I20260811 02:58:13.542026 19491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:13.563143 19491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:13.564100 19491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:13.564416 19491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:13.585661 19491 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:46511
I20260811 02:58:13.585706 19561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:46511 every 8 connection(s)
I20260811 02:58:13.587446 19491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:58:13.595394 19562 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:58:13.598061 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 19491
I20260811 02:58:13.598289 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/wal/instance
I20260811 02:58:13.599184 19562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: Bootstrap starting.
I20260811 02:58:13.621244 19562 log.cc:824] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:13.634258 19562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:58:13.634941 19562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: Bootstrap complete.
I20260811 02:58:13.643878 19562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } }
I20260811 02:58:13.644891 19562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92c330161dda44689dcfbde0de7f2fcf, State: Initialized, Role: FOLLOWER
I20260811 02:58:13.645538 19562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } }
I20260811 02:58:13.645874 19562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:13.646023 19562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:13.646238 19562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:58:13.650527 19562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } }
I20260811 02:58:13.651142 19562 leader_election.cc:302] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92c330161dda44689dcfbde0de7f2fcf; no voters:
I20260811 02:58:13.651902 19562 leader_election.cc:288] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:58:13.652043 19567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:58:13.652778 19567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [term 3 LEADER]: Becoming Leader. State: Replica: 92c330161dda44689dcfbde0de7f2fcf, State: Running, Role: LEADER
I20260811 02:58:13.653463 19567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } }
I20260811 02:58:13.654600 19562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:58:13.656368 19569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92c330161dda44689dcfbde0de7f2fcf. Latest consensus state: current_term: 3 leader_uuid: "92c330161dda44689dcfbde0de7f2fcf" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } }
I20260811 02:58:13.656715 19569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:13.657215 19568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "92c330161dda44689dcfbde0de7f2fcf" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "92c330161dda44689dcfbde0de7f2fcf" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 46511 } } }
I20260811 02:58:13.657634 19568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:13.660259 19574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:13.667102 19574 catalog_manager.cc:669] Loaded metadata for table first_table [id=07f1f8165f604852abe37e7bcaf72f34]
I20260811 02:58:13.669875 19574 tablet_loader.cc:96] loaded metadata for tablet a1e3d5a281884d0394daa5308da86ddd (table first_table [id=07f1f8165f604852abe37e7bcaf72f34])
I20260811 02:58:13.670450 19574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:58:13.672062 19574 catalog_manager.cc:1259] Loaded cluster ID: 5a7ed646581f489888943360f34589f4
I20260811 02:58:13.672176 19574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:13.676471 19574 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:13.678211 19574 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 92c330161dda44689dcfbde0de7f2fcf: Loaded TSK: 0
I20260811 02:58:13.680109 19574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:58:13.686252 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:0
--local_ip_for_outbound_sockets=127.13.94.129
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:46511
--builtin_ntp_servers=127.13.94.148:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260811 02:58:14.071408 19585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:14.071728 19585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:14.071869 19585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:14.079526 19585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:14.079751 19585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:58:14.089445 19585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:46511
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:58:14.090971 19585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:14.092921 19585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:14.107501 19593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:14.107577 19591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:14.107522 19590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:14.108419 19585 server_base.cc:1034] running on GCE node
I20260811 02:58:14.109167 19585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:14.110366 19585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:14.111641 19585 hybrid_clock.cc:648] HybridClock initialized: now 1786417094111541 us; error 78 us; skew 500 ppm
I20260811 02:58:14.112003 19585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:58:14.114724 19585 webserver.cc:460] Webserver started at http://127.13.94.129:44399/ using document root <none> and password file <none>
I20260811 02:58:14.115696 19585 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:14.115880 19585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:14.121670 19585 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:14.124944 19599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:14.126631 19585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:58:14.126855 19585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "62d5f705c166431a943d00e961f51819"
format_stamp: "Formatted at 2026-08-11 02:58:05 on dist-test-slave-p3t5"
I20260811 02:58:14.127949 19585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:14.149242 19585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:14.150226 19585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:14.150503 19585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:14.151537 19585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:14.153627 19606 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 02:58:14.159879 19585 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:58:14.160043 19585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260811 02:58:14.160210 19585 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:58:14.163513 19585 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:58:14.163635 19585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260811 02:58:14.164081 19606 tablet_bootstrap.cc:492] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Bootstrap starting.
I20260811 02:58:14.182565 19606 log.cc:824] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:14.187151 19606 tablet_bootstrap.cc:492] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:58:14.188069 19606 tablet_bootstrap.cc:492] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Bootstrap complete.
I20260811 02:58:14.189908 19606 ts_tablet_manager.cc:1397] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Time spent bootstrapping tablet: real 0.026s user 0.015s sys 0.010s
I20260811 02:58:14.203105 19606 raft_consensus.cc:348] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d5f705c166431a943d00e961f51819" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 36937 } }
I20260811 02:58:14.204730 19606 raft_consensus.cc:729] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d5f705c166431a943d00e961f51819, State: Initialized, Role: FOLLOWER
I20260811 02:58:14.205950 19606 consensus_queue.cc:260] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d5f705c166431a943d00e961f51819" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 36937 } }
I20260811 02:58:14.206274 19606 raft_consensus.cc:388] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:14.206461 19606 raft_consensus.cc:482] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:14.206668 19606 raft_consensus.cc:3037] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:58:14.211563 19606 raft_consensus.cc:504] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d5f705c166431a943d00e961f51819" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 36937 } }
I20260811 02:58:14.212352 19606 leader_election.cc:302] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62d5f705c166431a943d00e961f51819; no voters:
I20260811 02:58:14.213228 19606 leader_election.cc:288] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:58:14.213479 19704 raft_consensus.cc:2781] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:58:14.215056 19606 ts_tablet_manager.cc:1428] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819: Time spent starting tablet: real 0.025s user 0.022s sys 0.002s
I20260811 02:58:14.218039 19704 raft_consensus.cc:686] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [term 2 LEADER]: Becoming Leader. State: Replica: 62d5f705c166431a943d00e961f51819, State: Running, Role: LEADER
I20260811 02:58:14.218912 19704 consensus_queue.cc:237] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d5f705c166431a943d00e961f51819" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 36937 } }
I20260811 02:58:14.226475 19585 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:36941
I20260811 02:58:14.226512 19730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:36941 every 8 connection(s)
I20260811 02:58:14.227852 19585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:58:14.228315 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 19585
I20260811 02:58:14.228549 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance
I20260811 02:58:14.242290 19731 heartbeater.cc:347] Connected to a master server at 127.13.94.190:46511
I20260811 02:58:14.242709 19731 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:14.243637 19731 heartbeater.cc:511] Master 127.13.94.190:46511 requested a full tablet report, sending...
I20260811 02:58:14.247421 19523 ts_manager.cc:194] Registered new tserver with Master: 62d5f705c166431a943d00e961f51819 (127.13.94.129:36941)
I20260811 02:58:14.248706 19523 catalog_manager.cc:5495] T a1e3d5a281884d0394daa5308da86ddd P 62d5f705c166431a943d00e961f51819 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "62d5f705c166431a943d00e961f51819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d5f705c166431a943d00e961f51819" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:14.258134 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:58:14.262611 19523 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.94.129:39889
I20260811 02:58:14.266636 19731 heartbeater.cc:503] Master 127.13.94.190:46511 was elected leader, sending a full tablet report...
I20260811 02:58:14.356511 19656 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
92c330161dda44689dcfbde0de7f2fcf | 127.13.94.190:46511 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.13.94.148:35789 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
62d5f705c166431a943d00e961f51819 | 127.13.94.129:36941 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.13.94.129 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.13.94.148:35789 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 2 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 02:58:14.389182 13690 log_verifier.cc:126] Checking tablet a1e3d5a281884d0394daa5308da86ddd
I20260811 02:58:14.398142 13690 log_verifier.cc:177] Verified matching terms for 2 ops in tablet a1e3d5a281884d0394daa5308da86ddd
I20260811 02:58:14.427681 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 19585
I20260811 02:58:14.435851 19725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:58:14.531476 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 19585
I20260811 02:58:14.552872 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 19491
I20260811 02:58:14.554199 19557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 02:58:14.666198 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 19491
2026-08-11T02:58:14Z chronyd exiting
[ OK ] ParameterizedRemoveMasterTest.TestRemoveMaster/1 (11466 ms)
[----------] 1 test from ParameterizedRemoveMasterTest (11466 ms total)
[----------] 1 test from ParameterizedRecoverMasterTest
[ RUN ] ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy/1
2026-08-11T02:58:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:58:14Z Disabled control of system clock
I20260811 02:58:14.758888 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:39131
--webserver_interface=127.13.94.190
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:41759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:39131,127.13.94.189:39225,127.13.94.188:46089
--flush_threshold_secs=0
--log_segment_size_mb=1
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
W20260811 02:58:15.123890 19776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:15.124238 19776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:15.124413 19776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:15.132099 19776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:15.132230 19776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:15.132336 19776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:15.132442 19776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:58:15.132573 19776 flags.cc:425] Enabled experimental flag: --flush_threshold_secs=0
I20260811 02:58:15.142309 19776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:41759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:39131,127.13.94.189:39225,127.13.94.188:46089
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:39131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=0
--flush_threshold_secs=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:58:15.143816 19776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:15.145746 19776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:15.158762 19784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:15.158830 19782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:15.158806 19781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:15.159440 19776 server_base.cc:1034] running on GCE node
I20260811 02:58:15.160354 19776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:15.161957 19776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:15.163210 19776 hybrid_clock.cc:648] HybridClock initialized: now 1786417095163166 us; error 39 us; skew 500 ppm
I20260811 02:58:15.163676 19776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:58:15.166401 19776 webserver.cc:460] Webserver started at http://127.13.94.190:44035/ using document root <none> and password file <none>
I20260811 02:58:15.167269 19776 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:15.167428 19776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:15.167757 19776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:15.170627 19776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/data/instance:
uuid: "91c72b5787ad4f709936a56d6112a641"
format_stamp: "Formatted at 2026-08-11 02:58:15 on dist-test-slave-p3t5"
I20260811 02:58:15.171415 19776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/wal/instance:
uuid: "91c72b5787ad4f709936a56d6112a641"
format_stamp: "Formatted at 2026-08-11 02:58:15 on dist-test-slave-p3t5"
I20260811 02:58:15.176867 19776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 02:58:15.180639 19790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:15.182154 19776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:58:15.182307 19776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "91c72b5787ad4f709936a56d6112a641"
format_stamp: "Formatted at 2026-08-11 02:58:15 on dist-test-slave-p3t5"
I20260811 02:58:15.182509 19776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:15.199515 19776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:15.200454 19776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:15.200742 19776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:15.220497 19776 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:39131
I20260811 02:58:15.220505 19845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:39131 every 8 connection(s)
I20260811 02:58:15.222177 19776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:58:15.226212 19846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:15.229166 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 19776
I20260811 02:58:15.229408 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/wal/instance
I20260811 02:58:15.231133 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:39225
--webserver_interface=127.13.94.189
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:41759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:39131,127.13.94.189:39225,127.13.94.188:46089
--flush_threshold_secs=0
--log_segment_size_mb=1
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
I20260811 02:58:15.235044 19846 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:58:15.256402 19846 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } has no permanent_uuid. Determining permanent_uuid...
W20260811 02:58:15.257606 19793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.94.189:39225: connect: Connection refused (error 111)
W20260811 02:58:15.259372 19846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:39225: Network error: Client connection negotiation failed: client connection to 127.13.94.189:39225: connect: Connection refused (error 111)
I20260811 02:58:15.308840 19846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } attempt: 1
W20260811 02:58:15.311182 19846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:39225: Network error: Client connection negotiation failed: client connection to 127.13.94.189:39225: connect: Connection refused (error 111)
I20260811 02:58:15.379511 19846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } attempt: 2
W20260811 02:58:15.381726 19846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:39225: Network error: Client connection negotiation failed: client connection to 127.13.94.189:39225: connect: Connection refused (error 111)
I20260811 02:58:15.473075 19846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } attempt: 3
W20260811 02:58:15.475307 19846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:39225: Network error: Client connection negotiation failed: client connection to 127.13.94.189:39225: connect: Connection refused (error 111)
W20260811 02:58:15.605876 19850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:15.606236 19850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:15.606376 19850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:15.614576 19850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:15.614739 19850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:15.614820 19850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:15.614908 19850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:58:15.615018 19850 flags.cc:425] Enabled experimental flag: --flush_threshold_secs=0
I20260811 02:58:15.618650 19846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } attempt: 4
W20260811 02:58:15.621115 19846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.189:39225: Network error: Client connection negotiation failed: client connection to 127.13.94.189:39225: connect: Connection refused (error 111)
I20260811 02:58:15.627517 19850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:41759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:39131,127.13.94.189:39225,127.13.94.188:46089
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:39225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--webserver_port=0
--flush_threshold_secs=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:58:15.629535 19850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:15.632236 19850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:15.647872 19857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:15.647853 19856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:15.648020 19859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:15.649353 19850 server_base.cc:1034] running on GCE node
I20260811 02:58:15.650399 19850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:15.652226 19850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:15.653506 19850 hybrid_clock.cc:648] HybridClock initialized: now 1786417095653424 us; error 61 us; skew 500 ppm
I20260811 02:58:15.654006 19850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:58:15.657351 19850 webserver.cc:460] Webserver started at http://127.13.94.189:37743/ using document root <none> and password file <none>
I20260811 02:58:15.658432 19850 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:15.658568 19850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:15.658979 19850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:15.662806 19850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/data/instance:
uuid: "2484d71914e34bb3bbef9adbf21210a4"
format_stamp: "Formatted at 2026-08-11 02:58:15 on dist-test-slave-p3t5"
I20260811 02:58:15.663722 19850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/wal/instance:
uuid: "2484d71914e34bb3bbef9adbf21210a4"
format_stamp: "Formatted at 2026-08-11 02:58:15 on dist-test-slave-p3t5"
I20260811 02:58:15.671681 19850 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260811 02:58:15.676632 19865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:15.678798 19850 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:15.679080 19850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "2484d71914e34bb3bbef9adbf21210a4"
format_stamp: "Formatted at 2026-08-11 02:58:15 on dist-test-slave-p3t5"
I20260811 02:58:15.679404 19850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:15.703905 19850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:15.704888 19850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:15.705245 19850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:15.724888 19850 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:39225
I20260811 02:58:15.724897 19926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:39225 every 8 connection(s)
I20260811 02:58:15.726526 19850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:58:15.726915 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 19850
I20260811 02:58:15.727113 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/wal/instance
I20260811 02:58:15.728466 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:46089
--webserver_interface=127.13.94.188
--webserver_port=0
--builtin_ntp_servers=127.13.94.148:41759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:39131,127.13.94.189:39225,127.13.94.188:46089
--flush_threshold_secs=0
--log_segment_size_mb=1
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
I20260811 02:58:15.731050 19927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:15.737476 19927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:58:15.754577 19927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:58:15.765724 19927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } has no permanent_uuid. Determining permanent_uuid...
W20260811 02:58:15.767005 19868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.94.188:46089: connect: Connection refused (error 111)
W20260811 02:58:15.768939 19927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.188:46089: Network error: Client connection negotiation failed: client connection to 127.13.94.188:46089: connect: Connection refused (error 111)
I20260811 02:58:15.818341 19927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } attempt: 1
W20260811 02:58:15.820696 19927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.188:46089: Network error: Client connection negotiation failed: client connection to 127.13.94.188:46089: connect: Connection refused (error 111)
I20260811 02:58:15.889009 19927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } attempt: 2
W20260811 02:58:15.891649 19927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.188:46089: Network error: Client connection negotiation failed: client connection to 127.13.94.188:46089: connect: Connection refused (error 111)
I20260811 02:58:15.920439 19846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } attempt: 5
I20260811 02:58:15.927882 19846 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } has no permanent_uuid. Determining permanent_uuid...
W20260811 02:58:15.929893 19846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.188:46089: Network error: Client connection negotiation failed: client connection to 127.13.94.188:46089: connect: Connection refused (error 111)
I20260811 02:58:15.981215 19846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } attempt: 1
I20260811 02:58:15.982944 19927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } attempt: 3
W20260811 02:58:15.983862 19846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.188:46089: Network error: Client connection negotiation failed: client connection to 127.13.94.188:46089: connect: Connection refused (error 111)
W20260811 02:58:15.984891 19927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.188:46089: Network error: Client connection negotiation failed: client connection to 127.13.94.188:46089: connect: Connection refused (error 111)
I20260811 02:58:16.052186 19846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } attempt: 2
W20260811 02:58:16.054524 19846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.188:46089: Network error: Client connection negotiation failed: client connection to 127.13.94.188:46089: connect: Connection refused (error 111)
W20260811 02:58:16.097488 19929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:16.097831 19929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:16.097924 19929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:16.105357 19929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:16.105504 19929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:16.105549 19929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:16.105599 19929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:58:16.105746 19929 flags.cc:425] Enabled experimental flag: --flush_threshold_secs=0
I20260811 02:58:16.115574 19929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:41759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:39131,127.13.94.189:39225,127.13.94.188:46089
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.188:46089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.94.188
--webserver_port=0
--flush_threshold_secs=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:58:16.117093 19929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:16.119094 19929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:58:16.128144 19927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } attempt: 4
W20260811 02:58:16.130256 19927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.188:46089: Network error: Client connection negotiation failed: client connection to 127.13.94.188:46089: connect: Connection refused (error 111)
W20260811 02:58:16.132733 19939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:16.132865 19940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:16.132841 19942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:16.133692 19929 server_base.cc:1034] running on GCE node
I20260811 02:58:16.134631 19929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:16.136313 19929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:16.137537 19929 hybrid_clock.cc:648] HybridClock initialized: now 1786417096137466 us; error 62 us; skew 500 ppm
I20260811 02:58:16.138032 19929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:58:16.140520 19929 webserver.cc:460] Webserver started at http://127.13.94.188:44917/ using document root <none> and password file <none>
I20260811 02:58:16.141309 19929 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:16.141429 19929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:16.141860 19929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:16.144804 19929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/data/instance:
uuid: "49f332cd6dbe412892f57f5449b737a9"
format_stamp: "Formatted at 2026-08-11 02:58:16 on dist-test-slave-p3t5"
I20260811 02:58:16.145776 19929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/wal/instance:
uuid: "49f332cd6dbe412892f57f5449b737a9"
format_stamp: "Formatted at 2026-08-11 02:58:16 on dist-test-slave-p3t5"
I20260811 02:58:16.151662 19929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:16.155665 19949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:16.157287 19929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:58:16.157516 19929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/wal
uuid: "49f332cd6dbe412892f57f5449b737a9"
format_stamp: "Formatted at 2026-08-11 02:58:16 on dist-test-slave-p3t5"
I20260811 02:58:16.157836 19929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:16.160806 19846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } attempt: 3
W20260811 02:58:16.162940 19846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.94.188:46089: Network error: Client connection negotiation failed: client connection to 127.13.94.188:46089: connect: Connection refused (error 111)
I20260811 02:58:16.180743 19929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:16.181800 19929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:16.182137 19929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:16.201741 19929 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.188:46089
I20260811 02:58:16.201768 20014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.188:46089 every 8 connection(s)
I20260811 02:58:16.203236 19929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/data/info.pb
I20260811 02:58:16.207419 20015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:16.209082 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 19929
I20260811 02:58:16.209299 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/wal/instance
I20260811 02:58:16.215992 20015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:58:16.233964 20015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:58:16.241235 20015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:58:16.251933 20015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9: Bootstrap starting.
I20260811 02:58:16.255405 20015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:16.256518 20015 log.cc:824] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:16.259663 20015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9: No bootstrap required, opened a new log
I20260811 02:58:16.267416 20015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } }
I20260811 02:58:16.267959 20015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:16.268088 20015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f332cd6dbe412892f57f5449b737a9, State: Initialized, Role: FOLLOWER
I20260811 02:58:16.268740 20015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } }
I20260811 02:58:16.269685 20022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } } }
I20260811 02:58:16.270025 20022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:16.270804 20015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:58:16.279793 20033 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:58:16.279955 20033 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:58:16.289132 20022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:58:16.289510 20022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } }
I20260811 02:58:16.291376 20022 leader_election.cc:288] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c72b5787ad4f709936a56d6112a641 (127.13.94.190:39131), 2484d71914e34bb3bbef9adbf21210a4 (127.13.94.189:39225)
I20260811 02:58:16.340178 19846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } attempt: 4
I20260811 02:58:16.354696 19846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641: Bootstrap starting.
I20260811 02:58:16.359020 19846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:16.360469 19846 log.cc:824] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:16.363816 19846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641: No bootstrap required, opened a new log
I20260811 02:58:16.376121 19846 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } }
I20260811 02:58:16.376600 19846 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:16.376736 19846 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c72b5787ad4f709936a56d6112a641, State: Initialized, Role: FOLLOWER
I20260811 02:58:16.377492 19846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } }
I20260811 02:58:16.378942 20037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } } }
I20260811 02:58:16.379485 20037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:16.380466 19846 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:58:16.380491 19819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49f332cd6dbe412892f57f5449b737a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c72b5787ad4f709936a56d6112a641" is_pre_election: true
I20260811 02:58:16.381403 19819 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f332cd6dbe412892f57f5449b737a9 in term 0.
I20260811 02:58:16.383288 19952 leader_election.cc:302] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f332cd6dbe412892f57f5449b737a9, 91c72b5787ad4f709936a56d6112a641; no voters:
I20260811 02:58:16.384852 20022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:58:16.385075 20022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:58:16.385217 20022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:16.390941 20022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } }
I20260811 02:58:16.392910 19819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49f332cd6dbe412892f57f5449b737a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c72b5787ad4f709936a56d6112a641"
I20260811 02:58:16.393350 19819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:16.397646 19819 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f332cd6dbe412892f57f5449b737a9 in term 1.
I20260811 02:58:16.398810 20022 leader_election.cc:288] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [CANDIDATE]: Term 1 election: Requested vote from peers 91c72b5787ad4f709936a56d6112a641 (127.13.94.190:39131), 2484d71914e34bb3bbef9adbf21210a4 (127.13.94.189:39225)
I20260811 02:58:16.399662 19952 leader_election.cc:302] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f332cd6dbe412892f57f5449b737a9, 91c72b5787ad4f709936a56d6112a641; no voters:
I20260811 02:58:16.400182 20022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:16.400944 20022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [term 1 LEADER]: Becoming Leader. State: Replica: 49f332cd6dbe412892f57f5449b737a9, State: Running, Role: LEADER
I20260811 02:58:16.401870 20022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } }
I20260811 02:58:16.419957 20051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49f332cd6dbe412892f57f5449b737a9. Latest consensus state: current_term: 1 leader_uuid: "49f332cd6dbe412892f57f5449b737a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } } }
I20260811 02:58:16.420580 20051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:16.426347 20057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:16.428915 20057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:58:16.429464 19927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } attempt: 5
W20260811 02:58:16.429575 20058 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:58:16.429762 20058 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 02:58:16.434402 19955 tablet.cc:2369] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:58:16.447796 19927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4: Bootstrap starting.
I20260811 02:58:16.451457 19819 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [term 1 FOLLOWER]: Refusing update from remote peer 49f332cd6dbe412892f57f5449b737a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:16.452746 20022 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 02:58:16.453887 19796 tablet.cc:2369] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:58:16.455473 19927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:16.461500 19927 log.cc:824] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:16.478987 19927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4: No bootstrap required, opened a new log
I20260811 02:58:16.478964 20037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49f332cd6dbe412892f57f5449b737a9. Latest consensus state: current_term: 1 leader_uuid: "49f332cd6dbe412892f57f5449b737a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } } }
I20260811 02:58:16.479310 20037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:16.484413 20060 mvcc.cc:205] Tried to move back new op lower bound from 7317164427052015616 to 7317164426869313536. Current Snapshot: MvccSnapshot[applied={T|T < 7317164427052015616}]
I20260811 02:58:16.486349 20062 mvcc.cc:205] Tried to move back new op lower bound from 7317164427052015616 to 7317164426869313536. Current Snapshot: MvccSnapshot[applied={T|T < 7317164427052015616}]
I20260811 02:58:16.496240 20057 catalog_manager.cc:1347] Generated new cluster ID: 899386e8926c45ccb028ba14b7f9bcde
I20260811 02:58:16.496389 20057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:16.497131 20051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "49f332cd6dbe412892f57f5449b737a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } } }
I20260811 02:58:16.497524 20051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:16.511564 20037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "49f332cd6dbe412892f57f5449b737a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } } }
I20260811 02:58:16.511849 20037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:16.518548 19927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } }
I20260811 02:58:16.519086 19927 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:16.519222 19927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2484d71914e34bb3bbef9adbf21210a4, State: Initialized, Role: FOLLOWER
I20260811 02:58:16.519984 19927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } }
I20260811 02:58:16.521735 20068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } } }
I20260811 02:58:16.522850 20068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:16.527776 19927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:58:16.527835 19896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49f332cd6dbe412892f57f5449b737a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2484d71914e34bb3bbef9adbf21210a4" is_pre_election: true
I20260811 02:58:16.528537 19896 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f332cd6dbe412892f57f5449b737a9 in term 0.
I20260811 02:58:16.530120 19894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:16.533782 19894 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [term 1 FOLLOWER]: Refusing update from remote peer 49f332cd6dbe412892f57f5449b737a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:16.534128 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49f332cd6dbe412892f57f5449b737a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2484d71914e34bb3bbef9adbf21210a4"
I20260811 02:58:16.540004 20022 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:16.549962 20057 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:58:16.552057 20057 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:16.597285 20057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9: Generated new TSK 0
I20260811 02:58:16.598588 20057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:58:16.611243 20074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "49f332cd6dbe412892f57f5449b737a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } } }
I20260811 02:58:16.611451 20022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "49f332cd6dbe412892f57f5449b737a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } } }
I20260811 02:58:16.611759 20074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:16.611799 20022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:16.612134 20068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49f332cd6dbe412892f57f5449b737a9. Latest consensus state: current_term: 1 leader_uuid: "49f332cd6dbe412892f57f5449b737a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c72b5787ad4f709936a56d6112a641" member_type: VOTER last_known_addr { host: "127.13.94.190" port: 39131 } } peers { permanent_uuid: "2484d71914e34bb3bbef9adbf21210a4" member_type: VOTER last_known_addr { host: "127.13.94.189" port: 39225 } } peers { permanent_uuid: "49f332cd6dbe412892f57f5449b737a9" member_type: VOTER last_known_addr { host: "127.13.94.188" port: 46089 } } }
I20260811 02:58:16.612417 20068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:58:16.623821 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.94.129:0
--local_ip_for_outbound_sockets=127.13.94.129
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:39131,127.13.94.189:39225,127.13.94.188:46089
--builtin_ntp_servers=127.13.94.148:41759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260811 02:58:16.635527 20086 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 02:58:16.685909 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000116
I20260811 02:58:16.705411 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000091
I20260811 02:58:16.709095 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000051
I20260811 02:58:16.806303 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.119s user 0.033s sys 0.010s Metrics: {"bytes_written":6027,"cfile_init":1,"dirs.queue_time_us":19369,"dirs.run_cpu_time_us":327,"dirs.run_wall_time_us":7901,"drs_written":1,"lbm_read_time_us":206,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1311,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":3,"thread_start_us":19533,"threads_started":1}
I20260811 02:58:16.809693 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260811 02:58:16.810797 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4}
I20260811 02:58:16.844120 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.138s user 0.032s sys 0.028s Metrics: {"bytes_written":6027,"cfile_init":1,"dirs.queue_time_us":14497,"dirs.run_cpu_time_us":237,"dirs.run_wall_time_us":814,"drs_written":1,"lbm_read_time_us":157,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4093,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":3,"thread_start_us":14349,"threads_started":1}
I20260811 02:58:16.846154 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260811 02:58:16.852365 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.142s user 0.028s sys 0.018s Metrics: {"bytes_written":6027,"cfile_init":1,"compiler_manager_pool.queue_time_us":15596,"dirs.queue_time_us":5147,"dirs.run_cpu_time_us":287,"dirs.run_wall_time_us":5906,"drs_written":1,"lbm_read_time_us":134,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1443,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":3,"thread_start_us":35211,"threads_started":2}
I20260811 02:58:16.854321 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":125,"lbm_reads_lt_1ms":4}
I20260811 02:58:16.856091 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260811 02:58:16.857134 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":128,"lbm_reads_lt_1ms":4}
W20260811 02:58:17.154858 20087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:17.155247 20087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:17.155413 20087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:17.168231 20087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:17.168497 20087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.94.129
I20260811 02:58:17.179045 20087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:41759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.94.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.94.129
--webserver_port=0
--tserver_master_addrs=127.13.94.190:39131,127.13.94.189:39225,127.13.94.188:46089
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.94.129
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 02:58:17.180611 20087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:17.182825 20087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:17.200672 20099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:17.200703 20100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:17.201042 20102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:17.201856 20087 server_base.cc:1034] running on GCE node
I20260811 02:58:17.203074 20087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:17.204490 20087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:17.205777 20087 hybrid_clock.cc:648] HybridClock initialized: now 1786417097205669 us; error 65 us; skew 500 ppm
I20260811 02:58:17.206219 20087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 02:58:17.213994 20087 webserver.cc:460] Webserver started at http://127.13.94.129:44923/ using document root <none> and password file <none>
I20260811 02:58:17.215050 20087 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:17.215216 20087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:17.215615 20087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:17.219631 20087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/data/instance:
uuid: "1b2c3d6a808e479996d6dcca28229f02"
format_stamp: "Formatted at 2026-08-11 02:58:17 on dist-test-slave-p3t5"
I20260811 02:58:17.220827 20087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance:
uuid: "1b2c3d6a808e479996d6dcca28229f02"
format_stamp: "Formatted at 2026-08-11 02:58:17 on dist-test-slave-p3t5"
I20260811 02:58:17.229130 20087 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.007s
I20260811 02:58:17.234282 20109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:17.236228 20087 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:58:17.236445 20087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
uuid: "1b2c3d6a808e479996d6dcca28229f02"
format_stamp: "Formatted at 2026-08-11 02:58:17 on dist-test-slave-p3t5"
I20260811 02:58:17.236718 20087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:17.262600 20087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:17.263710 20087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:17.264081 20087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:17.265275 20087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:17.267081 20087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:17.267199 20087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:17.267374 20087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:17.267457 20087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:17.304700 20087 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.129:44519
I20260811 02:58:17.304742 20234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.129:44519 every 8 connection(s)
I20260811 02:58:17.306334 20087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/data/info.pb
I20260811 02:58:17.315891 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 20087
I20260811 02:58:17.316125 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/wal/instance
I20260811 02:58:17.332159 20236 heartbeater.cc:347] Connected to a master server at 127.13.94.188:46089
I20260811 02:58:17.332583 20236 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:17.333763 20236 heartbeater.cc:511] Master 127.13.94.188:46089 requested a full tablet report, sending...
I20260811 02:58:17.335937 20237 heartbeater.cc:347] Connected to a master server at 127.13.94.190:39131
I20260811 02:58:17.336098 20237 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:17.336378 19970 ts_manager.cc:194] Registered new tserver with Master: 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129:44519)
I20260811 02:58:17.336632 20237 heartbeater.cc:511] Master 127.13.94.190:39131 requested a full tablet report, sending...
I20260811 02:58:17.337690 20238 heartbeater.cc:347] Connected to a master server at 127.13.94.189:39225
I20260811 02:58:17.337836 20238 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:17.338207 20238 heartbeater.cc:511] Master 127.13.94.189:39225 requested a full tablet report, sending...
I20260811 02:58:17.338546 19970 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.94.129:54907
I20260811 02:58:17.338716 19809 ts_manager.cc:194] Registered new tserver with Master: 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129:44519)
I20260811 02:58:17.340396 19883 ts_manager.cc:194] Registered new tserver with Master: 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129:44519)
I20260811 02:58:17.350201 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:58:17.357995 13690 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260811 02:58:17.416509 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59042:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:17.444579 20058 catalog_manager.cc:1259] Loaded cluster ID: 899386e8926c45ccb028ba14b7f9bcde
I20260811 02:58:17.444682 20058 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641: loading cluster ID for follower catalog manager: success
I20260811 02:58:17.449051 20058 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641: acquiring CA information for follower catalog manager: success
I20260811 02:58:17.452672 20058 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:58:17.458200 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 3b6c4f5e21bc433d83c1084c3a06db13 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.1 [id=66c808d8308f471caa5924f5190af950]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:17.459599 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b6c4f5e21bc433d83c1084c3a06db13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:17.470587 20268 tablet_bootstrap.cc:492] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:17.474025 20268 tablet_bootstrap.cc:654] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:17.475381 20268 log.cc:824] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:17.478180 20268 tablet_bootstrap.cc:492] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:17.478417 20268 ts_tablet_manager.cc:1397] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260811 02:58:17.486480 20268 raft_consensus.cc:348] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.486903 20268 raft_consensus.cc:374] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:17.487046 20268 raft_consensus.cc:729] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:17.487660 20268 consensus_queue.cc:260] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.487882 20268 raft_consensus.cc:388] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:17.488001 20268 raft_consensus.cc:482] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:17.488134 20268 raft_consensus.cc:3037] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:17.490329 20268 raft_consensus.cc:504] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.490847 20268 leader_election.cc:302] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:17.491496 20268 leader_election.cc:288] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:17.491683 20271 raft_consensus.cc:2781] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:17.492434 20271 raft_consensus.cc:686] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:17.492673 20236 heartbeater.cc:503] Master 127.13.94.188:46089 was elected leader, sending a full tablet report...
I20260811 02:58:17.493085 20271 consensus_queue.cc:237] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.493508 20268 ts_tablet_manager.cc:1428] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 02:58:17.500237 19970 catalog_manager.cc:5495] T 3b6c4f5e21bc433d83c1084c3a06db13 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:17.537218 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59046:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:17.551700 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 7a797642abe4414e80cf82903a867347 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.2 [id=50f23dc0aa384999a63529c82071a9d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:17.552172 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a797642abe4414e80cf82903a867347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:17.557090 20268 tablet_bootstrap.cc:492] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:17.559068 20268 tablet_bootstrap.cc:654] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:17.561256 20268 tablet_bootstrap.cc:492] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:17.561465 20268 ts_tablet_manager.cc:1397] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:17.562600 20268 raft_consensus.cc:348] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.562840 20268 raft_consensus.cc:374] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:17.562866 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000210
I20260811 02:58:17.562945 20268 raft_consensus.cc:729] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:17.563186 20268 consensus_queue.cc:260] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.563452 20268 raft_consensus.cc:388] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:17.563560 20268 raft_consensus.cc:482] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:17.563674 20268 raft_consensus.cc:3037] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:17.565670 20268 raft_consensus.cc:504] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.565979 20268 leader_election.cc:302] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:17.566186 20268 leader_election.cc:288] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:17.566273 20271 raft_consensus.cc:2781] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:17.566497 20271 raft_consensus.cc:686] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:17.566790 20268 ts_tablet_manager.cc:1428] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260811 02:58:17.566720 20271 consensus_queue.cc:237] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.569664 19970 catalog_manager.cc:5495] T 7a797642abe4414e80cf82903a867347 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:17.579597 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.013s sys 0.000s Metrics: {"bytes_written":5589,"cfile_init":1,"dirs.queue_time_us":725,"dirs.run_cpu_time_us":339,"dirs.run_wall_time_us":3440,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2842,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":4,"thread_start_us":553,"threads_started":1}
I20260811 02:58:17.581127 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 426 bytes on disk
I20260811 02:58:17.582213 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4}
I20260811 02:58:17.582886 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260811 02:58:17.606556 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000212
I20260811 02:58:17.609082 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000210
I20260811 02:58:17.618937 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.024s sys 0.010s Metrics: {"bytes_written":6920,"cfile_cache_hit":12,"cfile_cache_hit_bytes":2239,"cfile_cache_miss":28,"cfile_cache_miss_bytes":3413,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1974,"dirs.run_cpu_time_us":1030,"dirs.run_wall_time_us":5232,"drs_written":1,"lbm_read_time_us":1032,"lbm_reads_lt_1ms":56,"lbm_write_time_us":882,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":50,"rows_written":7,"thread_start_us":8904,"threads_started":2}
I20260811 02:58:17.620268 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 475 bytes on disk
I20260811 02:58:17.621245 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4}
I20260811 02:58:17.629669 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.012s sys 0.001s Metrics: {"bytes_written":5626,"cfile_init":1,"dirs.queue_time_us":6827,"dirs.run_cpu_time_us":222,"dirs.run_wall_time_us":2365,"drs_written":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1542,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4,"thread_start_us":6656,"threads_started":1}
I20260811 02:58:17.630111 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.016s sys 0.003s Metrics: {"bytes_written":5626,"cfile_init":1,"dirs.queue_time_us":7246,"dirs.run_cpu_time_us":194,"dirs.run_wall_time_us":2260,"drs_written":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4,"lbm_write_time_us":980,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4,"thread_start_us":7089,"threads_started":1}
I20260811 02:58:17.631345 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260811 02:58:17.631487 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260811 02:58:17.631942 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59054:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:17.642016 20086 catalog_manager.cc:1259] Loaded cluster ID: 899386e8926c45ccb028ba14b7f9bcde
I20260811 02:58:17.642196 20086 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4: loading cluster ID for follower catalog manager: success
I20260811 02:58:17.648053 20086 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4: acquiring CA information for follower catalog manager: success
I20260811 02:58:17.652977 20086 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:58:17.657500 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf3dbdbc59b4fa1a564380141043ae5 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.3 [id=285b3f79930746ac95e0483a53ac6453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:17.658138 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf3dbdbc59b4fa1a564380141043ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:17.666905 20268 tablet_bootstrap.cc:492] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:17.668098 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.023s sys 0.010s Metrics: {"bytes_written":6920,"cfile_cache_hit":15,"cfile_cache_hit_bytes":2395,"cfile_cache_miss":25,"cfile_cache_miss_bytes":3355,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1699,"dirs.run_cpu_time_us":897,"dirs.run_wall_time_us":4217,"drs_written":1,"lbm_read_time_us":1217,"lbm_reads_lt_1ms":57,"lbm_write_time_us":1388,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":50,"rows_written":7,"thread_start_us":9933,"threads_started":2}
I20260811 02:58:17.668829 20268 tablet_bootstrap.cc:654] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:17.669230 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260811 02:58:17.670442 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.028s sys 0.007s Metrics: {"bytes_written":6920,"cfile_cache_hit":23,"cfile_cache_hit_bytes":3891,"cfile_cache_miss":17,"cfile_cache_miss_bytes":1859,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":3211,"dirs.run_cpu_time_us":1117,"dirs.run_wall_time_us":7275,"drs_written":1,"lbm_read_time_us":836,"lbm_reads_lt_1ms":37,"lbm_write_time_us":1424,"lbm_writes_lt_1ms":27,"mutex_wait_us":676,"num_input_rowsets":2,"peak_mem_usage":50,"rows_written":7,"spinlock_wait_cycles":2432,"thread_start_us":13106,"threads_started":3}
I20260811 02:58:17.671429 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 475 bytes on disk
I20260811 02:58:17.672610 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260811 02:58:17.673202 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260811 02:58:17.683166 20268 tablet_bootstrap.cc:492] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:17.683394 20268 ts_tablet_manager.cc:1397] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.007s
I20260811 02:58:17.684319 20268 raft_consensus.cc:348] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.684574 20268 raft_consensus.cc:374] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:17.684675 20268 raft_consensus.cc:729] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:17.685015 20268 consensus_queue.cc:260] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.685226 20268 raft_consensus.cc:388] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:17.685335 20268 raft_consensus.cc:482] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:17.685437 20268 raft_consensus.cc:3037] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:17.685964 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.012s sys 0.004s Metrics: {"bytes_written":5157,"cfile_init":1,"dirs.queue_time_us":95,"dirs.run_cpu_time_us":237,"dirs.run_wall_time_us":1198,"drs_written":1,"lbm_read_time_us":118,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1383,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:17.687034 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.370191
I20260811 02:58:17.687057 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.011s sys 0.000s Metrics: {"bytes_written":5157,"cfile_init":1,"dirs.queue_time_us":939,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":1277,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1171,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:17.687968 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:17.687994 20268 raft_consensus.cc:504] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.688341 20268 leader_election.cc:302] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:17.688577 20268 leader_election.cc:288] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:17.688643 20271 raft_consensus.cc:2781] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:17.688800 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":117,"lbm_reads_lt_1ms":4}
I20260811 02:58:17.689313 20271 raft_consensus.cc:686] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:17.689605 20271 consensus_queue.cc:237] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.689671 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.370191
I20260811 02:58:17.694195 20268 ts_tablet_manager.cc:1428] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.011s user 0.003s sys 0.003s
I20260811 02:58:17.699674 19970 catalog_manager.cc:5495] T 9cf3dbdbc59b4fa1a564380141043ae5 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:17.714216 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.022s sys 0.004s Metrics: {"bytes_written":7350,"cfile_cache_hit":12,"cfile_cache_hit_bytes":2610,"cfile_cache_miss":28,"cfile_cache_miss_bytes":3970,"cfile_init":9,"delta_iterators_relevant":4,"dirs.queue_time_us":901,"dirs.run_cpu_time_us":739,"dirs.run_wall_time_us":4646,"drs_written":1,"lbm_read_time_us":1489,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1138,"lbm_writes_lt_1ms":27,"mutex_wait_us":24,"num_input_rowsets":2,"peak_mem_usage":561,"rows_written":9,"spinlock_wait_cycles":10880,"thread_start_us":2694,"threads_started":1}
I20260811 02:58:17.715348 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260811 02:58:17.721871 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.005s sys 0.000s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":176,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:17.722543 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 542 bytes on disk
I20260811 02:58:17.723122 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260811 02:58:17.723745 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.121014
I20260811 02:58:17.727953 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.028s sys 0.000s Metrics: {"bytes_written":334,"cfile_cache_hit":12,"cfile_cache_hit_bytes":2610,"cfile_cache_miss":28,"cfile_cache_miss_bytes":3970,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":823,"dirs.run_cpu_time_us":674,"dirs.run_wall_time_us":6240,"drs_written":1,"lbm_read_time_us":1274,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1443,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":561,"reinsert_count":0,"rows_written":9,"thread_start_us":5393,"threads_started":1,"update_count":1}
I20260811 02:58:17.729146 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.121014
I20260811 02:58:17.739089 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.004s sys 0.008s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2428,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":100,"lbm_read_time_us":415,"lbm_reads_lt_1ms":17,"lbm_write_time_us":354,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:17.739893 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000004
I20260811 02:58:17.742744 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2428,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":135,"lbm_read_time_us":507,"lbm_reads_lt_1ms":21,"lbm_write_time_us":337,"lbm_writes_lt_1ms":9,"mutex_wait_us":30,"reinsert_count":0,"spinlock_wait_cycles":896,"update_count":1}
I20260811 02:58:17.743789 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000006
I20260811 02:58:17.746034 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.003s sys 0.002s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":151,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:17.746587 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.120616
I20260811 02:58:17.749964 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.005s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":177,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:17.750517 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.120616
I20260811 02:58:17.756745 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.004s sys 0.005s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2465,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":134,"lbm_read_time_us":427,"lbm_reads_lt_1ms":25,"lbm_write_time_us":324,"lbm_writes_lt_1ms":9,"mutex_wait_us":32,"reinsert_count":0,"spinlock_wait_cycles":17792,"update_count":1}
I20260811 02:58:17.760993 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.006s sys 0.003s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2465,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":85,"lbm_read_time_us":473,"lbm_reads_lt_1ms":25,"lbm_write_time_us":420,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:17.761822 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 542 bytes on disk
I20260811 02:58:17.762574 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4}
I20260811 02:58:17.775344 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59058:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:17.788944 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 4158169129b842afbc2eeb73c4ea7bc5 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.4 [id=c1d16b0d7e2c4c09bb08321100759418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:17.789492 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4158169129b842afbc2eeb73c4ea7bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:17.794345 20268 tablet_bootstrap.cc:492] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:17.796015 20268 tablet_bootstrap.cc:654] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:17.798429 20268 tablet_bootstrap.cc:492] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:17.798596 20268 ts_tablet_manager.cc:1397] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:17.799245 20268 raft_consensus.cc:348] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.799407 20268 raft_consensus.cc:374] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:17.799536 20268 raft_consensus.cc:729] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:17.799803 20268 consensus_queue.cc:260] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.800019 20268 raft_consensus.cc:388] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:17.800107 20268 raft_consensus.cc:482] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:17.800237 20268 raft_consensus.cc:3037] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:17.802348 20268 raft_consensus.cc:504] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.802664 20268 leader_election.cc:302] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:17.802868 20268 leader_election.cc:288] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:17.802944 20271 raft_consensus.cc:2781] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:17.803158 20271 raft_consensus.cc:686] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:17.803395 20271 consensus_queue.cc:237] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.803922 20268 ts_tablet_manager.cc:1428] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:17.806433 19968 catalog_manager.cc:5495] T 4158169129b842afbc2eeb73c4ea7bc5 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:17.841660 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59062:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:17.854813 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce2b5532af24527ae4226e1807d42bf (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.5 [id=a72c06c1b50a4479842fc9d44d983d05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:17.855347 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce2b5532af24527ae4226e1807d42bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:17.860625 20268 tablet_bootstrap.cc:492] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:17.862355 20268 tablet_bootstrap.cc:654] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:17.864358 20268 tablet_bootstrap.cc:492] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:17.864566 20268 ts_tablet_manager.cc:1397] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:17.865229 20268 raft_consensus.cc:348] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.865489 20268 raft_consensus.cc:374] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:17.865598 20268 raft_consensus.cc:729] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:17.865890 20268 consensus_queue.cc:260] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.866077 20268 raft_consensus.cc:388] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:17.866206 20268 raft_consensus.cc:482] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:17.866308 20268 raft_consensus.cc:3037] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:17.868271 20268 raft_consensus.cc:504] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.868543 20268 leader_election.cc:302] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:17.868773 20268 leader_election.cc:288] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:17.868844 20271 raft_consensus.cc:2781] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:17.869047 20271 raft_consensus.cc:686] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:17.869302 20271 consensus_queue.cc:237] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.869608 20268 ts_tablet_manager.cc:1428] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:17.872362 19968 catalog_manager.cc:5495] T 8ce2b5532af24527ae4226e1807d42bf P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:17.872917 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:17.887282 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.009s sys 0.004s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":1907,"drs_written":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1162,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:17.888259 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.532085
I20260811 02:58:17.911963 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.020s sys 0.002s Metrics: {"bytes_written":8262,"cfile_cache_hit":13,"cfile_cache_hit_bytes":3378,"cfile_cache_miss":31,"cfile_cache_miss_bytes":5736,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":2582,"dirs.run_cpu_time_us":737,"dirs.run_wall_time_us":9063,"drs_written":1,"lbm_read_time_us":1214,"lbm_reads_lt_1ms":67,"lbm_write_time_us":971,"lbm_writes_lt_1ms":27,"mutex_wait_us":35,"num_input_rowsets":2,"peak_mem_usage":561,"rows_written":13,"thread_start_us":2467,"threads_started":1}
I20260811 02:58:17.912988 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 742 bytes on disk
I20260811 02:58:17.913774 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":122,"lbm_reads_lt_1ms":4}
I20260811 02:58:17.922662 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59078:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:17.936010 20165 tablet_service.cc:1467] Processing CreateTablet for tablet d346af9c140345639f5bcb0a7b556c0e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.6 [id=5d8806bd5cbf4d1ba9e93bb7c1426199]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:17.936534 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d346af9c140345639f5bcb0a7b556c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:17.941468 20268 tablet_bootstrap.cc:492] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:17.943190 20268 tablet_bootstrap.cc:654] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:17.945483 20268 tablet_bootstrap.cc:492] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:17.945684 20268 ts_tablet_manager.cc:1397] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:58:17.946360 20268 raft_consensus.cc:348] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.946523 20268 raft_consensus.cc:374] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:17.946669 20268 raft_consensus.cc:729] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:17.946910 20268 consensus_queue.cc:260] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.947108 20268 raft_consensus.cc:388] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:17.947208 20268 raft_consensus.cc:482] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:17.947361 20268 raft_consensus.cc:3037] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:17.949323 20268 raft_consensus.cc:504] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.949589 20268 leader_election.cc:302] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:17.949815 20268 leader_election.cc:288] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:17.949915 20271 raft_consensus.cc:2781] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:17.950138 20271 raft_consensus.cc:686] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:17.950340 20268 ts_tablet_manager.cc:1428] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:17.950377 20271 consensus_queue.cc:237] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:17.953691 19968 catalog_manager.cc:5495] T d346af9c140345639f5bcb0a7b556c0e P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:17.991593 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59080:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.005805 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 5c3311a7a9744dd995598ae7a602f65a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.7 [id=f922c672ef78400d8ec3ab417cb6d984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.006300 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c3311a7a9744dd995598ae7a602f65a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.007866 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000089
I20260811 02:58:18.011164 20268 tablet_bootstrap.cc:492] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.013044 20268 tablet_bootstrap.cc:654] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.014472 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000091
I20260811 02:58:18.015841 20268 tablet_bootstrap.cc:492] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.016057 20268 ts_tablet_manager.cc:1397] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260811 02:58:18.016885 20268 raft_consensus.cc:348] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.017163 20268 raft_consensus.cc:374] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.017248 20268 raft_consensus.cc:729] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.017539 20268 consensus_queue.cc:260] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.017779 20268 raft_consensus.cc:388] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.017879 20268 raft_consensus.cc:482] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.017983 20268 raft_consensus.cc:3037] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.020757 20268 raft_consensus.cc:504] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.021080 20268 leader_election.cc:302] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.020980 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.000s Metrics: {"bytes_written":6459,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":250,"dirs.run_wall_time_us":1577,"drs_written":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4,"lbm_write_time_us":905,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260811 02:58:18.021314 20268 leader_election.cc:288] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.021404 20271 raft_consensus.cc:2781] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.021596 20271 raft_consensus.cc:686] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.021878 20271 consensus_queue.cc:237] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.022047 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 586 bytes on disk
I20260811 02:58:18.022576 20268 ts_tablet_manager.cc:1428] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 02:58:18.023392 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.024180 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.524701
I20260811 02:58:18.025007 19968 catalog_manager.cc:5495] T 5c3311a7a9744dd995598ae7a602f65a P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.028530 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.010s sys 0.001s Metrics: {"bytes_written":6459,"cfile_init":1,"dirs.queue_time_us":75,"dirs.run_cpu_time_us":191,"dirs.run_wall_time_us":1096,"drs_written":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4,"lbm_write_time_us":734,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260811 02:58:18.029551 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 586 bytes on disk
I20260811 02:58:18.031674 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.032456 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.524701
I20260811 02:58:18.050057 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.019s sys 0.005s Metrics: {"bytes_written":9119,"cfile_cache_hit":14,"cfile_cache_hit_bytes":4146,"cfile_cache_miss":34,"cfile_cache_miss_bytes":7180,"cfile_init":9,"delta_iterators_relevant":8,"dirs.queue_time_us":1200,"dirs.run_cpu_time_us":651,"dirs.run_wall_time_us":3629,"drs_written":1,"lbm_read_time_us":1603,"lbm_reads_lt_1ms":70,"lbm_write_time_us":1040,"lbm_writes_lt_1ms":27,"mutex_wait_us":173,"num_input_rowsets":2,"peak_mem_usage":1053,"rows_written":17,"thread_start_us":886,"threads_started":2}
I20260811 02:58:18.051079 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 890 bytes on disk
I20260811 02:58:18.051751 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.052364 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260811 02:58:18.057083 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.023s sys 0.000s Metrics: {"bytes_written":334,"cfile_cache_hit":14,"cfile_cache_hit_bytes":4146,"cfile_cache_miss":34,"cfile_cache_miss_bytes":7180,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":8,"dirs.queue_time_us":864,"dirs.run_cpu_time_us":895,"dirs.run_wall_time_us":5468,"drs_written":1,"lbm_read_time_us":1440,"lbm_reads_lt_1ms":70,"lbm_write_time_us":1094,"lbm_writes_lt_1ms":31,"mutex_wait_us":16,"num_input_rowsets":2,"peak_mem_usage":1053,"reinsert_count":0,"rows_written":17,"thread_start_us":557,"threads_started":1,"update_count":1}
I20260811 02:58:18.058135 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 890 bytes on disk
I20260811 02:58:18.058037 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.003s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":155,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:18.058563 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260811 02:58:18.060835 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.061511 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260811 02:58:18.063233 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.002s sys 0.001s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":130,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:18.063704 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.157734
I20260811 02:58:18.067528 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":159,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:18.068060 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.157734
I20260811 02:58:18.074831 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.001s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":3874,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":112,"lbm_read_time_us":399,"lbm_reads_lt_1ms":27,"lbm_write_time_us":262,"lbm_writes_lt_1ms":9,"mutex_wait_us":1,"reinsert_count":0,"update_count":2}
I20260811 02:58:18.075508 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260811 02:58:18.075780 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260811 02:58:18.077090 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.005s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":3874,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":117,"lbm_read_time_us":366,"lbm_reads_lt_1ms":23,"lbm_write_time_us":230,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":2}
I20260811 02:58:18.077734 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260811 02:58:18.077996 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260811 02:58:18.096087 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59088:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.109481 20165 tablet_service.cc:1467] Processing CreateTablet for tablet cb954462e54f44db9ccf417a4241b124 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.8 [id=0db0ffd726d54332b73160d994d6c9bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.110057 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb954462e54f44db9ccf417a4241b124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.115049 20268 tablet_bootstrap.cc:492] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.116431 20268 tablet_bootstrap.cc:654] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.118621 20268 tablet_bootstrap.cc:492] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.118798 20268 ts_tablet_manager.cc:1397] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:18.119462 20268 raft_consensus.cc:348] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.119678 20268 raft_consensus.cc:374] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.119809 20268 raft_consensus.cc:729] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.120082 20268 consensus_queue.cc:260] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.120282 20268 raft_consensus.cc:388] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.120385 20268 raft_consensus.cc:482] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.120496 20268 raft_consensus.cc:3037] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.123153 20268 raft_consensus.cc:504] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.123447 20268 leader_election.cc:302] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.123660 20268 leader_election.cc:288] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.123744 20271 raft_consensus.cc:2781] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.123971 20271 raft_consensus.cc:686] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.124200 20268 ts_tablet_manager.cc:1428] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:18.124231 20271 consensus_queue.cc:237] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.127024 19968 catalog_manager.cc:5495] T cb954462e54f44db9ccf417a4241b124 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.164235 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59090:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.164891 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260811 02:58:18.177701 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 2c30125f85cf4864b53a1602b5a72d87 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.9 [id=1d58fb472d6c4b3d89322096b3997af9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.178200 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c30125f85cf4864b53a1602b5a72d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.180037 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.004s sys 0.008s Metrics: {"bytes_written":6066,"cfile_init":1,"dirs.queue_time_us":38,"dirs.run_cpu_time_us":191,"dirs.run_wall_time_us":1728,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1082,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:18.180898 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 545 bytes on disk
I20260811 02:58:18.181536 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.182296 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.445830
I20260811 02:58:18.184993 20268 tablet_bootstrap.cc:492] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.186822 20268 tablet_bootstrap.cc:654] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.193040 20268 tablet_bootstrap.cc:492] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.193251 20268 ts_tablet_manager.cc:1397] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 02:58:18.194205 20268 raft_consensus.cc:348] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.194475 20268 raft_consensus.cc:374] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.194566 20268 raft_consensus.cc:729] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.194826 20268 consensus_queue.cc:260] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.195041 20268 raft_consensus.cc:388] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.195155 20268 raft_consensus.cc:482] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.195258 20268 raft_consensus.cc:3037] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.198304 20268 raft_consensus.cc:504] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.198594 20268 leader_election.cc:302] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.198828 20268 leader_election.cc:288] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.198931 20271 raft_consensus.cc:2781] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.199123 20271 raft_consensus.cc:686] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.199383 20271 consensus_queue.cc:237] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.202732 20268 ts_tablet_manager.cc:1428] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260811 02:58:18.205920 19968 catalog_manager.cc:5495] T 2c30125f85cf4864b53a1602b5a72d87 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.233393 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.051s user 0.023s sys 0.012s Metrics: {"bytes_written":9588,"cfile_cache_hit":12,"cfile_cache_hit_bytes":4492,"cfile_cache_miss":28,"cfile_cache_miss_bytes":7056,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1215,"dirs.run_cpu_time_us":734,"dirs.run_wall_time_us":5149,"drs_written":1,"lbm_read_time_us":1398,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1028,"lbm_writes_lt_1ms":27,"mutex_wait_us":79,"num_input_rowsets":2,"peak_mem_usage":1531,"rows_written":19,"spinlock_wait_cycles":13568,"thread_start_us":25504,"threads_started":1}
I20260811 02:58:18.234622 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1002 bytes on disk
I20260811 02:58:18.235455 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.236877 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000016
I20260811 02:58:18.248052 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":5194,"cfile_init":1,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":176,"dirs.run_wall_time_us":1293,"drs_written":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4,"lbm_write_time_us":862,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:18.248840 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.294421
I20260811 02:58:18.269227 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59106:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.271593 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.021s sys 0.000s Metrics: {"bytes_written":10031,"cfile_cache_hit":12,"cfile_cache_hit_bytes":4861,"cfile_cache_miss":28,"cfile_cache_miss_bytes":7655,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1563,"dirs.run_cpu_time_us":1034,"dirs.run_wall_time_us":4986,"drs_written":1,"lbm_read_time_us":1207,"lbm_reads_lt_1ms":60,"lbm_write_time_us":946,"lbm_writes_lt_1ms":27,"mutex_wait_us":417,"num_input_rowsets":2,"peak_mem_usage":2507,"rows_written":21,"thread_start_us":868,"threads_started":2}
I20260811 02:58:18.272627 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1087 bytes on disk
I20260811 02:58:18.273267 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.283238 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 6b4aa349f62046979059e02a6b8b22db (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.10 [id=b5e756d6b60d48d59609c1c91677a94f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.283722 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b4aa349f62046979059e02a6b8b22db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.288770 20268 tablet_bootstrap.cc:492] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.290392 20268 tablet_bootstrap.cc:654] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.292515 20268 tablet_bootstrap.cc:492] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.292650 20268 ts_tablet_manager.cc:1397] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:18.293279 20268 raft_consensus.cc:348] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.293444 20268 raft_consensus.cc:374] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.293510 20268 raft_consensus.cc:729] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.293777 20268 consensus_queue.cc:260] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.293955 20268 raft_consensus.cc:388] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.294018 20268 raft_consensus.cc:482] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.294099 20268 raft_consensus.cc:3037] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.296473 20268 raft_consensus.cc:504] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.296785 20268 leader_election.cc:302] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.296983 20268 leader_election.cc:288] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.297067 20271 raft_consensus.cc:2781] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.297266 20271 raft_consensus.cc:686] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.297572 20268 ts_tablet_manager.cc:1428] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260811 02:58:18.297544 20271 consensus_queue.cc:237] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.301267 19968 catalog_manager.cc:5495] T 6b4aa349f62046979059e02a6b8b22db P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.326583 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260811 02:58:18.328943 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260811 02:58:18.341552 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.004s sys 0.004s Metrics: {"bytes_written":6067,"cfile_init":1,"dirs.queue_time_us":63,"dirs.run_cpu_time_us":99,"dirs.run_wall_time_us":1306,"drs_written":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4,"lbm_write_time_us":842,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:18.342447 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.008s sys 0.006s Metrics: {"bytes_written":6067,"cfile_init":1,"dirs.queue_time_us":64,"dirs.run_cpu_time_us":203,"dirs.run_wall_time_us":1380,"drs_written":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1229,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:18.342780 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 551 bytes on disk
I20260811 02:58:18.343416 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.414504
I20260811 02:58:18.343767 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":122,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.344535 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.414504
I20260811 02:58:18.367532 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.018s sys 0.005s Metrics: {"bytes_written":10464,"cfile_cache_hit":13,"cfile_cache_hit_bytes":5265,"cfile_cache_miss":31,"cfile_cache_miss_bytes":8605,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":422,"dirs.run_cpu_time_us":896,"dirs.run_wall_time_us":4216,"drs_written":1,"lbm_read_time_us":1395,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1094,"lbm_writes_lt_1ms":27,"mutex_wait_us":69,"num_input_rowsets":2,"peak_mem_usage":2262,"rows_written":23,"spinlock_wait_cycles":1408}
I20260811 02:58:18.368677 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1172 bytes on disk
I20260811 02:58:18.369395 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.378765 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.018s sys 0.004s Metrics: {"bytes_written":10464,"cfile_cache_hit":13,"cfile_cache_hit_bytes":5265,"cfile_cache_miss":31,"cfile_cache_miss_bytes":8605,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":2005,"dirs.run_cpu_time_us":1001,"dirs.run_wall_time_us":4659,"drs_written":1,"lbm_read_time_us":1258,"lbm_reads_lt_1ms":63,"lbm_write_time_us":979,"lbm_writes_lt_1ms":27,"mutex_wait_us":23,"num_input_rowsets":2,"peak_mem_usage":2262,"rows_written":23}
I20260811 02:58:18.379802 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1172 bytes on disk
I20260811 02:58:18.381088 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.385900 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59112:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.399823 20165 tablet_service.cc:1467] Processing CreateTablet for tablet e5ea5fec7b3f480f82383ed27006e900 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.11 [id=26845af4f1824f9a91cd7b289c4da847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.400418 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ea5fec7b3f480f82383ed27006e900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.405259 20268 tablet_bootstrap.cc:492] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.406713 20268 tablet_bootstrap.cc:654] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.409050 20268 tablet_bootstrap.cc:492] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.409243 20268 ts_tablet_manager.cc:1397] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:18.409926 20268 raft_consensus.cc:348] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.410173 20268 raft_consensus.cc:374] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.410264 20268 raft_consensus.cc:729] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.410552 20268 consensus_queue.cc:260] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.410733 20268 raft_consensus.cc:388] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.410848 20268 raft_consensus.cc:482] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.410966 20268 raft_consensus.cc:3037] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.413357 20268 raft_consensus.cc:504] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.413671 20268 leader_election.cc:302] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.413909 20268 leader_election.cc:288] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.414000 20271 raft_consensus.cc:2781] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.414143 20271 raft_consensus.cc:686] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.414397 20271 consensus_queue.cc:237] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.414976 20268 ts_tablet_manager.cc:1428] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:58:18.417662 19970 catalog_manager.cc:5495] T e5ea5fec7b3f480f82383ed27006e900 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.463233 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59120:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.475986 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 7d2ad0f243b34be79a19dfb8777d8639 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.12 [id=42b4b13bbd1945959fee70a2cb3f2ff0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.476449 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d2ad0f243b34be79a19dfb8777d8639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.481841 20268 tablet_bootstrap.cc:492] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.483294 20268 tablet_bootstrap.cc:654] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.485404 20268 tablet_bootstrap.cc:492] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.485556 20268 ts_tablet_manager.cc:1397] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:18.486246 20268 raft_consensus.cc:348] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.486426 20268 raft_consensus.cc:374] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.486496 20268 raft_consensus.cc:729] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.486722 20268 consensus_queue.cc:260] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.486894 20268 raft_consensus.cc:388] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.486963 20268 raft_consensus.cc:482] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.487035 20268 raft_consensus.cc:3037] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.489022 20268 raft_consensus.cc:504] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.489259 20268 leader_election.cc:302] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.489445 20268 leader_election.cc:288] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.489526 20271 raft_consensus.cc:2781] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.489784 20271 raft_consensus.cc:686] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.489976 20268 ts_tablet_manager.cc:1428] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:58:18.490036 20271 consensus_queue.cc:237] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.493490 19970 catalog_manager.cc:5495] T 7d2ad0f243b34be79a19dfb8777d8639 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.524688 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260811 02:58:18.530473 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59128:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.537762 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.002s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":76,"dirs.run_cpu_time_us":212,"dirs.run_wall_time_us":1427,"drs_written":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4,"lbm_write_time_us":912,"lbm_writes_lt_1ms":27,"mutex_wait_us":3,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:18.538800 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 549 bytes on disk
I20260811 02:58:18.539359 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.540042 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.393358
I20260811 02:58:18.543398 20165 tablet_service.cc:1467] Processing CreateTablet for tablet f4c36774fddf47de92d1e1cd33b33786 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.13 [id=9a26b9b7bea64f0f8754fe43fa45583b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.543952 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c36774fddf47de92d1e1cd33b33786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.548990 20268 tablet_bootstrap.cc:492] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.550772 20268 tablet_bootstrap.cc:654] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.553278 20268 tablet_bootstrap.cc:492] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.553484 20268 ts_tablet_manager.cc:1397] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:18.554155 20268 raft_consensus.cc:348] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.554363 20268 raft_consensus.cc:374] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.554468 20268 raft_consensus.cc:729] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.554749 20268 consensus_queue.cc:260] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.554944 20268 raft_consensus.cc:388] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.555070 20268 raft_consensus.cc:482] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.555172 20268 raft_consensus.cc:3037] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.558873 20268 raft_consensus.cc:504] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.559192 20268 leader_election.cc:302] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.559453 20268 leader_election.cc:288] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.559569 20271 raft_consensus.cc:2781] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.559743 20271 raft_consensus.cc:686] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.560029 20271 consensus_queue.cc:237] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.560786 20268 ts_tablet_manager.cc:1428] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260811 02:58:18.566416 19970 catalog_manager.cc:5495] T f4c36774fddf47de92d1e1cd33b33786 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.570489 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.021s sys 0.000s Metrics: {"bytes_written":11357,"cfile_cache_hit":12,"cfile_cache_hit_bytes":5971,"cfile_cache_miss":28,"cfile_cache_miss_bytes":9267,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1040,"dirs.run_cpu_time_us":219,"dirs.run_wall_time_us":1291,"drs_written":1,"lbm_read_time_us":2478,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":55,"lbm_write_time_us":897,"lbm_writes_lt_1ms":27,"mutex_wait_us":43,"num_input_rowsets":2,"peak_mem_usage":2967,"rows_written":27,"spinlock_wait_cycles":222208}
I20260811 02:58:18.571497 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260811 02:58:18.586109 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.013s sys 0.000s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":1300,"drs_written":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1083,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:18.587108 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1591 bytes on disk
I20260811 02:58:18.587827 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":110,"lbm_reads_lt_1ms":8}
I20260811 02:58:18.588449 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.459198
I20260811 02:58:18.597308 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.004s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":576,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":137,"lbm_read_time_us":320,"lbm_reads_lt_1ms":17,"lbm_write_time_us":308,"lbm_writes_lt_1ms":9,"mutex_wait_us":22,"reinsert_count":0,"spinlock_wait_cycles":896,"update_count":1}
I20260811 02:58:18.598030 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.323168
I20260811 02:58:18.620433 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260811 02:58:18.621129 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.018s sys 0.003s Metrics: {"bytes_written":11800,"cfile_cache_hit":13,"cfile_cache_hit_bytes":6371,"cfile_cache_miss":31,"cfile_cache_miss_bytes":10009,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1229,"dirs.run_cpu_time_us":1150,"dirs.run_wall_time_us":10051,"drs_written":1,"lbm_read_time_us":970,"lbm_reads_lt_1ms":63,"lbm_write_time_us":705,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":3937,"rows_written":29,"thread_start_us":1992,"threads_started":1}
I20260811 02:58:18.626250 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1450 bytes on disk
I20260811 02:58:18.627013 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.630318 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59144:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.632361 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:18.632648 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.002s sys 0.007s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":255,"dirs.run_wall_time_us":1292,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":679,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6,"spinlock_wait_cycles":3840}
I20260811 02:58:18.643713 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.384053
I20260811 02:58:18.645385 20165 tablet_service.cc:1467] Processing CreateTablet for tablet fdc7942472da4040bd21914665cd87a8 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.14 [id=5599c5fcd3524110b926aa0b7043dfdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.645931 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdc7942472da4040bd21914665cd87a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.645754 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.013s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1713,"drs_written":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1029,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:18.647789 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.384053
I20260811 02:58:18.652796 20268 tablet_bootstrap.cc:492] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.654632 20268 tablet_bootstrap.cc:654] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.656985 20268 tablet_bootstrap.cc:492] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.657258 20268 ts_tablet_manager.cc:1397] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 02:58:18.658324 20268 raft_consensus.cc:348] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.658650 20268 raft_consensus.cc:374] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.658766 20268 raft_consensus.cc:729] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.659039 20268 consensus_queue.cc:260] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.659281 20268 raft_consensus.cc:388] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.659384 20268 raft_consensus.cc:482] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.659533 20268 raft_consensus.cc:3037] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.662498 20268 raft_consensus.cc:504] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.662835 20268 leader_election.cc:302] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.663106 20268 leader_election.cc:288] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.663221 20271 raft_consensus.cc:2781] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.663410 20271 raft_consensus.cc:686] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.663664 20271 consensus_queue.cc:237] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.664542 20268 ts_tablet_manager.cc:1428] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:58:18.667796 19970 catalog_manager.cc:5495] T fdc7942472da4040bd21914665cd87a8 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.676107 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.010s sys 0.007s Metrics: {"bytes_written":11800,"cfile_cache_hit":12,"cfile_cache_hit_bytes":6336,"cfile_cache_miss":28,"cfile_cache_miss_bytes":9828,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":55,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":895,"drs_written":1,"lbm_read_time_us":1003,"lbm_reads_lt_1ms":60,"lbm_write_time_us":921,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":3222,"rows_written":29}
I20260811 02:58:18.677083 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260811 02:58:18.686157 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.021s sys 0.007s Metrics: {"bytes_written":11800,"cfile_cache_hit":12,"cfile_cache_hit_bytes":6336,"cfile_cache_miss":28,"cfile_cache_miss_bytes":9828,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":6824,"dirs.run_cpu_time_us":1027,"dirs.run_wall_time_us":5647,"drs_written":1,"lbm_read_time_us":1218,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1003,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":3222,"rows_written":29,"thread_start_us":9001,"threads_started":1}
I20260811 02:58:18.689977 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1450 bytes on disk
I20260811 02:58:18.690650 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.691260 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000016
I20260811 02:58:18.692297 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.007s sys 0.004s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":253,"dirs.run_wall_time_us":1174,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2709,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:18.693181 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.291522
I20260811 02:58:18.703104 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.002s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":199,"dirs.run_wall_time_us":1124,"drs_written":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1583,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:18.704063 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 359 bytes on disk
I20260811 02:58:18.704638 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.705345 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.291522
I20260811 02:58:18.738338 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.029s sys 0.000s Metrics: {"bytes_written":12231,"cfile_cache_hit":12,"cfile_cache_hit_bytes":6716,"cfile_cache_miss":28,"cfile_cache_miss_bytes":10396,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":567,"dirs.run_cpu_time_us":1042,"dirs.run_wall_time_us":5559,"drs_written":1,"lbm_read_time_us":2712,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":55,"lbm_write_time_us":1251,"lbm_writes_lt_1ms":27,"mutex_wait_us":119,"num_input_rowsets":2,"peak_mem_usage":4397,"rows_written":31,"spinlock_wait_cycles":17152}
I20260811 02:58:18.739827 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1530 bytes on disk
I20260811 02:58:18.740893 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":145,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.745820 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.052s user 0.037s sys 0.012s Metrics: {"bytes_written":12231,"cfile_cache_hit":12,"cfile_cache_hit_bytes":6716,"cfile_cache_miss":28,"cfile_cache_miss_bytes":10396,"cfile_init":9,"delta_iterators_relevant":4,"dirs.queue_time_us":505,"dirs.run_cpu_time_us":837,"dirs.run_wall_time_us":4084,"drs_written":1,"lbm_read_time_us":1117,"lbm_reads_lt_1ms":64,"lbm_write_time_us":987,"lbm_writes_lt_1ms":27,"mutex_wait_us":45,"num_input_rowsets":2,"peak_mem_usage":4397,"rows_written":31,"spinlock_wait_cycles":6016,"thread_start_us":27866,"threads_started":1}
I20260811 02:58:18.746937 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1530 bytes on disk
I20260811 02:58:18.747702 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":116,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.751580 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59150:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.765779 20165 tablet_service.cc:1467] Processing CreateTablet for tablet f2e99cca357a42338d48d5d9a7239d60 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.15 [id=87881cf1fee1436e86ce470767107a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.766249 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2e99cca357a42338d48d5d9a7239d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.770576 20268 tablet_bootstrap.cc:492] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.772248 20268 tablet_bootstrap.cc:654] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.774554 20268 tablet_bootstrap.cc:492] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.774740 20268 ts_tablet_manager.cc:1397] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:58:18.775357 20268 raft_consensus.cc:348] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.775568 20268 raft_consensus.cc:374] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.775656 20268 raft_consensus.cc:729] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.775874 20268 consensus_queue.cc:260] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.776018 20268 raft_consensus.cc:388] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.776157 20268 raft_consensus.cc:482] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.776257 20268 raft_consensus.cc:3037] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.778999 20268 raft_consensus.cc:504] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.779328 20268 leader_election.cc:302] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.779577 20268 leader_election.cc:288] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.779645 20271 raft_consensus.cc:2781] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.779860 20271 raft_consensus.cc:686] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.780242 20268 ts_tablet_manager.cc:1428] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:18.780146 20271 consensus_queue.cc:237] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.783521 19970 catalog_manager.cc:5495] T f2e99cca357a42338d48d5d9a7239d60 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.820392 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59160:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.834455 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 5d17741f303f4a6b90d19db88ac0a0da (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.16 [id=6d93148a96394308af7fff23bfb196b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.834955 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d17741f303f4a6b90d19db88ac0a0da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.839733 20268 tablet_bootstrap.cc:492] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.841144 20268 tablet_bootstrap.cc:654] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.843323 20268 tablet_bootstrap.cc:492] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.843485 20268 ts_tablet_manager.cc:1397] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:18.844245 20268 raft_consensus.cc:348] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.844422 20268 raft_consensus.cc:374] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.844539 20268 raft_consensus.cc:729] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.844789 20268 consensus_queue.cc:260] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.844980 20268 raft_consensus.cc:388] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.845081 20268 raft_consensus.cc:482] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.845192 20268 raft_consensus.cc:3037] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.847182 20268 raft_consensus.cc:504] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.847471 20268 leader_election.cc:302] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.847679 20268 leader_election.cc:288] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.848001 20273 raft_consensus.cc:2781] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.848244 20273 raft_consensus.cc:686] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.848251 20268 ts_tablet_manager.cc:1428] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260811 02:58:18.848605 20273 consensus_queue.cc:237] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.851966 19970 catalog_manager.cc:5495] T 5d17741f303f4a6b90d19db88ac0a0da P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.878099 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:18.891791 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.011s sys 0.000s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":212,"dirs.run_wall_time_us":2200,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1137,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:18.892589 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.357882
I20260811 02:58:18.896240 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59166:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:18.913357 20165 tablet_service.cc:1467] Processing CreateTablet for tablet c8ca0b6702cf42b599f490f0c689ee79 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.17 [id=5e6b934cf8f14539b3f6f55625d36529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:18.913841 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ca0b6702cf42b599f490f0c689ee79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:18.915179 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.018s sys 0.004s Metrics: {"bytes_written":13127,"cfile_cache_hit":12,"cfile_cache_hit_bytes":7451,"cfile_cache_miss":28,"cfile_cache_miss_bytes":11449,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":767,"dirs.run_cpu_time_us":195,"dirs.run_wall_time_us":1839,"drs_written":1,"lbm_read_time_us":1024,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1183,"lbm_writes_lt_1ms":27,"mutex_wait_us":67,"num_input_rowsets":2,"peak_mem_usage":4397,"rows_written":35,"spinlock_wait_cycles":4736}
I20260811 02:58:18.916265 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1697 bytes on disk
I20260811 02:58:18.916854 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.917389 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000007
I20260811 02:58:18.927714 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.004s sys 0.004s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":109,"dirs.run_cpu_time_us":251,"dirs.run_wall_time_us":1272,"drs_written":1,"lbm_read_time_us":69,"lbm_reads_lt_1ms":4,"lbm_write_time_us":624,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:18.928517 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:18.929184 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.929813 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.275079
I20260811 02:58:18.948017 20268 tablet_bootstrap.cc:492] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:18.948190 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.017s sys 0.000s Metrics: {"bytes_written":13534,"cfile_cache_hit":12,"cfile_cache_hit_bytes":7783,"cfile_cache_miss":28,"cfile_cache_miss_bytes":11753,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":359,"dirs.run_cpu_time_us":235,"dirs.run_wall_time_us":1245,"drs_written":1,"lbm_read_time_us":846,"lbm_reads_lt_1ms":56,"lbm_write_time_us":826,"lbm_writes_lt_1ms":27,"mutex_wait_us":44,"num_input_rowsets":2,"peak_mem_usage":5162,"rows_written":37,"spinlock_wait_cycles":1280}
I20260811 02:58:18.949266 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1770 bytes on disk
I20260811 02:58:18.949894 20268 tablet_bootstrap.cc:654] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:18.950238 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20260811 02:58:18.951864 20268 tablet_bootstrap.cc:492] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:18.952059 20268 ts_tablet_manager.cc:1397] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:18.952721 20268 raft_consensus.cc:348] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.952950 20268 raft_consensus.cc:374] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:18.953044 20268 raft_consensus.cc:729] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:18.953337 20268 consensus_queue.cc:260] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.953528 20268 raft_consensus.cc:388] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:18.953667 20268 raft_consensus.cc:482] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:18.953792 20268 raft_consensus.cc:3037] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:18.956171 20268 raft_consensus.cc:504] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.956435 20268 leader_election.cc:302] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:18.956655 20268 leader_election.cc:288] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:18.956852 20273 raft_consensus.cc:2781] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:18.957073 20273 raft_consensus.cc:686] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:18.957290 20273 consensus_queue.cc:237] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:18.957405 20268 ts_tablet_manager.cc:1428] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260811 02:58:18.960800 19970 catalog_manager.cc:5495] T c8ca0b6702cf42b599f490f0c689ee79 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:18.992146 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:18.999207 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260811 02:58:19.005821 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.006s sys 0.007s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":188,"dirs.run_wall_time_us":1327,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":783,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:19.007027 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.351132
I20260811 02:58:19.014523 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.013s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":199,"dirs.run_wall_time_us":1720,"drs_written":1,"lbm_read_time_us":139,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1093,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:19.015476 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 519 bytes on disk
I20260811 02:58:19.016191 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.016896 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.351132
I20260811 02:58:19.018179 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59170:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.033842 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 481619d9e1164d10b47625226df4e546 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.18 [id=6ce2003f2ff648fe9410159f760abe0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.034399 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481619d9e1164d10b47625226df4e546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.036492 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.024s sys 0.001s Metrics: {"bytes_written":13571,"cfile_cache_hit":12,"cfile_cache_hit_bytes":7816,"cfile_cache_miss":28,"cfile_cache_miss_bytes":11976,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1077,"dirs.run_cpu_time_us":903,"dirs.run_wall_time_us":7877,"drs_written":1,"lbm_read_time_us":1339,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1566,"lbm_writes_lt_1ms":27,"mutex_wait_us":94,"num_input_rowsets":2,"peak_mem_usage":4652,"rows_written":37,"thread_start_us":982,"threads_started":1}
I20260811 02:58:19.037770 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260811 02:58:19.040824 20268 tablet_bootstrap.cc:492] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.042734 20268 tablet_bootstrap.cc:654] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.043071 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.012s sys 0.011s Metrics: {"bytes_written":13571,"cfile_cache_hit":12,"cfile_cache_hit_bytes":7816,"cfile_cache_miss":28,"cfile_cache_miss_bytes":11976,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":940,"dirs.run_cpu_time_us":907,"dirs.run_wall_time_us":7193,"drs_written":1,"lbm_read_time_us":1082,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1036,"lbm_writes_lt_1ms":27,"mutex_wait_us":158,"num_input_rowsets":2,"peak_mem_usage":4652,"rows_written":37,"spinlock_wait_cycles":1280,"thread_start_us":1115,"threads_started":1}
I20260811 02:58:19.044171 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000008
I20260811 02:58:19.045609 20268 tablet_bootstrap.cc:492] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.045835 20268 ts_tablet_manager.cc:1397] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:19.046716 20268 raft_consensus.cc:348] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.046968 20268 raft_consensus.cc:374] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.047079 20268 raft_consensus.cc:729] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.047391 20268 consensus_queue.cc:260] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.047624 20268 raft_consensus.cc:388] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.047711 20268 raft_consensus.cc:482] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.047837 20268 raft_consensus.cc:3037] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.050582 20268 raft_consensus.cc:504] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.050890 20268 leader_election.cc:302] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.051175 20273 raft_consensus.cc:2781] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.051329 20273 raft_consensus.cc:686] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.051621 20273 consensus_queue.cc:237] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.052466 20268 leader_election.cc:288] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.053254 20268 ts_tablet_manager.cc:1428] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260811 02:58:19.056005 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.012s sys 0.000s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":77,"dirs.run_cpu_time_us":188,"dirs.run_wall_time_us":2002,"drs_written":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1216,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:19.057207 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1949 bytes on disk
I20260811 02:58:19.057873 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.011s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":73,"dirs.run_cpu_time_us":222,"dirs.run_wall_time_us":1106,"drs_written":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4,"lbm_write_time_us":864,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":768}
I20260811 02:58:19.058115 19970 catalog_manager.cc:5495] T 481619d9e1164d10b47625226df4e546 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.058339 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":189,"lbm_reads_lt_1ms":8}
I20260811 02:58:19.058808 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2017 bytes on disk
I20260811 02:58:19.059167 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260811 02:58:19.060114 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":170,"lbm_reads_lt_1ms":8}
I20260811 02:58:19.060899 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.244340
I20260811 02:58:19.072505 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":300,"lbm_read_time_us":2405,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":20,"lbm_write_time_us":317,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:19.073208 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.262904
I20260811 02:58:19.091003 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.016s sys 0.008s Metrics: {"bytes_written":390,"cfile_cache_hit":12,"cfile_cache_hit_bytes":8158,"cfile_cache_miss":28,"cfile_cache_miss_bytes":12298,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":710,"dirs.run_cpu_time_us":210,"dirs.run_wall_time_us":1173,"drs_written":1,"lbm_read_time_us":1120,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1261,"lbm_writes_lt_1ms":31,"mutex_wait_us":25,"num_input_rowsets":2,"peak_mem_usage":5417,"reinsert_count":0,"rows_written":39,"spinlock_wait_cycles":10624,"update_count":2}
I20260811 02:58:19.092155 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1852 bytes on disk
I20260811 02:58:19.092937 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.098893 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.025s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":8191,"cfile_cache_miss":31,"cfile_cache_miss_bytes":12367,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1132,"dirs.run_cpu_time_us":597,"dirs.run_wall_time_us":2841,"drs_written":1,"lbm_read_time_us":1271,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1224,"lbm_writes_lt_1ms":31,"mutex_wait_us":73,"num_input_rowsets":2,"peak_mem_usage":5417,"reinsert_count":0,"rows_written":39,"spinlock_wait_cycles":896,"thread_start_us":373,"threads_started":1,"update_count":1}
I20260811 02:58:19.099737 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1862 bytes on disk
I20260811 02:58:19.100308 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.118721 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59186:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.132764 20165 tablet_service.cc:1467] Processing CreateTablet for tablet b7f40809285f4b0f88be6b66c4a0acb7 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.19 [id=a045a908d115448b8d19d74bdc50a968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.133294 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7f40809285f4b0f88be6b66c4a0acb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.138266 20268 tablet_bootstrap.cc:492] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.139842 20268 tablet_bootstrap.cc:654] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.142107 20268 tablet_bootstrap.cc:492] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.142282 20268 ts_tablet_manager.cc:1397] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:19.143014 20268 raft_consensus.cc:348] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.143235 20268 raft_consensus.cc:374] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.143342 20268 raft_consensus.cc:729] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.143620 20268 consensus_queue.cc:260] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.143769 20268 raft_consensus.cc:388] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.143893 20268 raft_consensus.cc:482] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.144013 20268 raft_consensus.cc:3037] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.146193 20268 raft_consensus.cc:504] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.146448 20268 leader_election.cc:302] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.146693 20268 leader_election.cc:288] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.146847 20273 raft_consensus.cc:2781] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.147063 20273 raft_consensus.cc:686] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.147207 20268 ts_tablet_manager.cc:1428] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260811 02:58:19.147297 20273 consensus_queue.cc:237] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.150606 19968 catalog_manager.cc:5495] T b7f40809285f4b0f88be6b66c4a0acb7 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.189342 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59196:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.201361 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000076
I20260811 02:58:19.201818 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 72ef961adc524b55ba6d6b787377dd4a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.20 [id=ec90231ccf0d46038605c3208451ca6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.202332 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72ef961adc524b55ba6d6b787377dd4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.210570 20268 tablet_bootstrap.cc:492] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.212267 20268 tablet_bootstrap.cc:654] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.214351 20268 tablet_bootstrap.cc:492] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.214540 20268 ts_tablet_manager.cc:1397] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:19.215286 20268 raft_consensus.cc:348] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.215481 20268 raft_consensus.cc:374] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.215583 20268 raft_consensus.cc:729] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.215865 20268 consensus_queue.cc:260] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.216068 20268 raft_consensus.cc:388] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.216173 20268 raft_consensus.cc:482] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.216348 20268 raft_consensus.cc:3037] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.218825 20268 raft_consensus.cc:504] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.218725 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.002s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":90,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1237,"drs_written":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1488,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:19.219122 20268 leader_election.cc:302] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.219427 20273 raft_consensus.cc:2781] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.219595 20273 raft_consensus.cc:686] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.219779 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 529 bytes on disk
I20260811 02:58:19.219906 20273 consensus_queue.cc:237] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.223728 20268 leader_election.cc:288] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.226300 20268 ts_tablet_manager.cc:1428] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260811 02:58:19.230625 19968 catalog_manager.cc:5495] T 72ef961adc524b55ba6d6b787377dd4a P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.238270 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":57,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.239032 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.329141
I20260811 02:58:19.272972 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.023s sys 0.001s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":8857,"cfile_cache_miss":28,"cfile_cache_miss_bytes":13557,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":536,"dirs.run_cpu_time_us":1015,"dirs.run_wall_time_us":5889,"drs_written":1,"lbm_read_time_us":1264,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1102,"lbm_writes_lt_1ms":31,"mutex_wait_us":73,"num_input_rowsets":2,"peak_mem_usage":5246,"reinsert_count":0,"rows_written":43,"update_count":1}
I20260811 02:58:19.274384 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2046 bytes on disk
I20260811 02:58:19.275473 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":137,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.287133 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59202:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.300218 20165 tablet_service.cc:1467] Processing CreateTablet for tablet d6cce80397084963af6a87924d25e634 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.21 [id=3a4d683260a94689a36ccb67d79a66c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.300632 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6cce80397084963af6a87924d25e634. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.305781 20268 tablet_bootstrap.cc:492] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.307183 20268 tablet_bootstrap.cc:654] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.309536 20268 tablet_bootstrap.cc:492] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.309763 20268 ts_tablet_manager.cc:1397] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:19.310456 20268 raft_consensus.cc:348] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.310667 20268 raft_consensus.cc:374] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.310762 20268 raft_consensus.cc:729] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.311080 20268 consensus_queue.cc:260] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.311271 20268 raft_consensus.cc:388] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.311391 20268 raft_consensus.cc:482] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.311509 20268 raft_consensus.cc:3037] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.313844 20268 raft_consensus.cc:504] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.314105 20268 leader_election.cc:302] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.314391 20268 leader_election.cc:288] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.314502 20273 raft_consensus.cc:2781] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.314685 20273 raft_consensus.cc:686] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.314914 20268 ts_tablet_manager.cc:1428] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260811 02:58:19.314918 20273 consensus_queue.cc:237] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
W20260811 02:58:19.316645 20239 tablet.cc:2369] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:58:19.325276 19968 catalog_manager.cc:5495] T d6cce80397084963af6a87924d25e634 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.344363 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:19.351264 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260811 02:58:19.356084 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.003s sys 0.005s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":76,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1379,"drs_written":1,"lbm_read_time_us":54,"lbm_reads_lt_1ms":4,"lbm_write_time_us":631,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:19.357100 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 550 bytes on disk
I20260811 02:58:19.357836 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":130,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.358579 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.325081
I20260811 02:58:19.364986 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":152,"dirs.run_wall_time_us":1051,"drs_written":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4,"lbm_write_time_us":946,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:19.366005 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.325300
I20260811 02:58:19.386179 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.025s sys 0.000s Metrics: {"bytes_written":15322,"cfile_cache_hit":13,"cfile_cache_hit_bytes":9293,"cfile_cache_miss":31,"cfile_cache_miss_bytes":14709,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":357,"dirs.run_cpu_time_us":726,"dirs.run_wall_time_us":4660,"drs_written":1,"lbm_read_time_us":1272,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1372,"lbm_writes_lt_1ms":27,"mutex_wait_us":201,"num_input_rowsets":2,"peak_mem_usage":5929,"rows_written":45}
I20260811 02:58:19.387193 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2148 bytes on disk
I20260811 02:58:19.387812 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.393004 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.024s sys 0.000s Metrics: {"bytes_written":15322,"cfile_cache_hit":13,"cfile_cache_hit_bytes":9327,"cfile_cache_miss":31,"cfile_cache_miss_bytes":14657,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1500,"dirs.run_cpu_time_us":207,"dirs.run_wall_time_us":1152,"drs_written":1,"lbm_read_time_us":1533,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1196,"lbm_writes_lt_1ms":27,"mutex_wait_us":73,"num_input_rowsets":2,"peak_mem_usage":5896,"rows_written":45}
I20260811 02:58:19.393973 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2148 bytes on disk
I20260811 02:58:19.394625 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.396981 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59216:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.410251 20165 tablet_service.cc:1467] Processing CreateTablet for tablet bae13cff46894575875b95b3f56712d6 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.22 [id=2b7d2e34e2c9437cb4f919cffa59dffa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.410740 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bae13cff46894575875b95b3f56712d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.415555 20268 tablet_bootstrap.cc:492] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.417002 20268 tablet_bootstrap.cc:654] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.419116 20268 tablet_bootstrap.cc:492] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.419303 20268 ts_tablet_manager.cc:1397] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:19.419981 20268 raft_consensus.cc:348] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.420200 20268 raft_consensus.cc:374] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.420323 20268 raft_consensus.cc:729] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.420581 20268 consensus_queue.cc:260] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.420783 20268 raft_consensus.cc:388] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.420885 20268 raft_consensus.cc:482] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.421020 20268 raft_consensus.cc:3037] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.423357 20268 raft_consensus.cc:504] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.423648 20268 leader_election.cc:302] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.423856 20268 leader_election.cc:288] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.424027 20273 raft_consensus.cc:2781] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.424217 20273 raft_consensus.cc:686] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.424396 20268 ts_tablet_manager.cc:1428] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:19.424499 20273 consensus_queue.cc:237] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.428933 19968 catalog_manager.cc:5495] T bae13cff46894575875b95b3f56712d6 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.471607 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59220:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.485725 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 69e2be7533184cef9f3c0b0cdad6fe8b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.23 [id=0d4eeba3238148148d2b175281eecaa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.486163 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69e2be7533184cef9f3c0b0cdad6fe8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.490715 20268 tablet_bootstrap.cc:492] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.492105 20268 tablet_bootstrap.cc:654] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.494103 20268 tablet_bootstrap.cc:492] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.494282 20268 ts_tablet_manager.cc:1397] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:19.494935 20268 raft_consensus.cc:348] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.495148 20268 raft_consensus.cc:374] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.495240 20268 raft_consensus.cc:729] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.495585 20268 consensus_queue.cc:260] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.495806 20268 raft_consensus.cc:388] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.495929 20268 raft_consensus.cc:482] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.496037 20268 raft_consensus.cc:3037] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.498210 20268 raft_consensus.cc:504] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.498476 20268 leader_election.cc:302] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.498690 20268 leader_election.cc:288] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.498891 20273 raft_consensus.cc:2781] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.499119 20273 raft_consensus.cc:686] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.499166 20268 ts_tablet_manager.cc:1428] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260811 02:58:19.499359 20273 consensus_queue.cc:237] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.503185 19968 catalog_manager.cc:5495] T 69e2be7533184cef9f3c0b0cdad6fe8b P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.527220 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000086
I20260811 02:58:19.541074 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":257,"dirs.run_wall_time_us":1316,"drs_written":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4,"lbm_write_time_us":952,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:19.542076 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.318904
I20260811 02:58:19.545989 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59232:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.561359 20165 tablet_service.cc:1467] Processing CreateTablet for tablet b82f4601bcc54b6c9859ec6af7fbcf0c (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.24 [id=280d6be1401a49ea8e27e81544efdfc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.561997 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b82f4601bcc54b6c9859ec6af7fbcf0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.570101 20268 tablet_bootstrap.cc:492] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.569787 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.022s sys 0.003s Metrics: {"bytes_written":16206,"cfile_cache_hit":13,"cfile_cache_hit_bytes":10067,"cfile_cache_miss":31,"cfile_cache_miss_bytes":15719,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":994,"dirs.run_cpu_time_us":253,"dirs.run_wall_time_us":2007,"drs_written":1,"lbm_read_time_us":1376,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1157,"lbm_writes_lt_1ms":27,"mutex_wait_us":310,"num_input_rowsets":2,"peak_mem_usage":6616,"rows_written":49}
I20260811 02:58:19.570686 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2313 bytes on disk
I20260811 02:58:19.571223 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.571841 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260811 02:58:19.571882 20268 tablet_bootstrap.cc:654] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.574066 20268 tablet_bootstrap.cc:492] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.574224 20268 ts_tablet_manager.cc:1397] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:19.574882 20268 raft_consensus.cc:348] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.575045 20268 raft_consensus.cc:374] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.575155 20268 raft_consensus.cc:729] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.575408 20268 consensus_queue.cc:260] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.575662 20268 raft_consensus.cc:388] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.575752 20268 raft_consensus.cc:482] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.575867 20268 raft_consensus.cc:3037] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.578313 20268 raft_consensus.cc:504] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.578590 20268 leader_election.cc:302] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.578804 20268 leader_election.cc:288] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.578907 20273 raft_consensus.cc:2781] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.579057 20273 raft_consensus.cc:686] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.579339 20273 consensus_queue.cc:237] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.579511 20268 ts_tablet_manager.cc:1428] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260811 02:58:19.583585 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.007s sys 0.004s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":39,"dirs.run_cpu_time_us":158,"dirs.run_wall_time_us":1265,"drs_written":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4,"lbm_write_time_us":830,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:19.583623 19968 catalog_manager.cc:5495] T b82f4601bcc54b6c9859ec6af7fbcf0c P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.584645 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:19.585384 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.586149 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.259761
I20260811 02:58:19.609598 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.021s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":10378,"cfile_cache_miss":28,"cfile_cache_miss_bytes":15606,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1496,"dirs.run_cpu_time_us":164,"dirs.run_wall_time_us":1262,"drs_written":1,"lbm_read_time_us":1150,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1222,"lbm_writes_lt_1ms":31,"mutex_wait_us":51,"num_input_rowsets":2,"peak_mem_usage":7363,"reinsert_count":0,"rows_written":51,"update_count":1}
I20260811 02:58:19.610541 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2380 bytes on disk
I20260811 02:58:19.611105 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.628758 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59234:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.639081 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:19.642336 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 6208ac5acf7d4dce92838c826c698d7b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.25 [id=5bbfbd48b8ad407391edde6c4276f058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.642851 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6208ac5acf7d4dce92838c826c698d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.645720 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260811 02:58:19.652714 20268 tablet_bootstrap.cc:492] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.654723 20268 tablet_bootstrap.cc:654] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.654843 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.010s sys 0.003s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":116,"dirs.run_cpu_time_us":261,"dirs.run_wall_time_us":1873,"drs_written":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1172,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260811 02:58:19.656181 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.350228
I20260811 02:58:19.657891 20268 tablet_bootstrap.cc:492] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.658133 20268 ts_tablet_manager.cc:1397] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260811 02:58:19.658871 20268 raft_consensus.cc:348] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.659116 20268 raft_consensus.cc:374] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.659206 20268 raft_consensus.cc:729] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.659480 20268 consensus_queue.cc:260] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.659710 20268 raft_consensus.cc:388] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.659821 20268 raft_consensus.cc:482] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.659955 20268 raft_consensus.cc:3037] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.660835 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.011s sys 0.000s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":84,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1205,"drs_written":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1109,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260811 02:58:19.661964 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 535 bytes on disk
I20260811 02:58:19.662642 20268 raft_consensus.cc:504] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.662734 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.662925 20268 leader_election.cc:302] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.663213 20268 leader_election.cc:288] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.663300 20273 raft_consensus.cc:2781] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.663502 20273 raft_consensus.cc:686] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.663544 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.350228
I20260811 02:58:19.663749 20273 consensus_queue.cc:237] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.664143 20268 ts_tablet_manager.cc:1428] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:58:19.667296 19968 catalog_manager.cc:5495] T 6208ac5acf7d4dce92838c826c698d7b P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.698869 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.022s sys 0.007s Metrics: {"bytes_written":390,"cfile_cache_hit":13,"cfile_cache_hit_bytes":10657,"cfile_cache_miss":31,"cfile_cache_miss_bytes":16523,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1054,"dirs.run_cpu_time_us":976,"dirs.run_wall_time_us":5508,"drs_written":1,"lbm_read_time_us":3461,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":62,"lbm_write_time_us":2970,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"mutex_wait_us":21,"num_input_rowsets":2,"peak_mem_usage":6852,"reinsert_count":0,"rows_written":53,"thread_start_us":523,"threads_started":1,"update_count":2}
I20260811 02:58:19.699798 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.025s sys 0.004s Metrics: {"bytes_written":334,"cfile_cache_hit":13,"cfile_cache_hit_bytes":10657,"cfile_cache_miss":31,"cfile_cache_miss_bytes":16523,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1360,"dirs.run_cpu_time_us":1140,"dirs.run_wall_time_us":9352,"drs_written":1,"lbm_read_time_us":1387,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1437,"lbm_writes_lt_1ms":31,"mutex_wait_us":239,"num_input_rowsets":2,"peak_mem_usage":6852,"reinsert_count":0,"rows_written":53,"thread_start_us":3930,"threads_started":2,"update_count":1}
I20260811 02:58:19.700812 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260811 02:58:19.701082 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2464 bytes on disk
I20260811 02:58:19.702201 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.706090 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.003s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":165,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:19.706662 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2464 bytes on disk
I20260811 02:58:19.707383 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.708016 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260811 02:58:19.716029 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.003s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":88,"lbm_read_time_us":100,"lbm_reads_lt_1ms":8,"lbm_write_time_us":168,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260811 02:58:19.729176 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59242:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.745093 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 4b1d1031995f4f8d901b51e896034647 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.26 [id=44a46c64d31740448009231902cae7e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.745687 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b1d1031995f4f8d901b51e896034647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.752775 20268 tablet_bootstrap.cc:492] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.754436 20268 tablet_bootstrap.cc:654] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.756333 20268 tablet_bootstrap.cc:492] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.756494 20268 ts_tablet_manager.cc:1397] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:19.757376 20268 raft_consensus.cc:348] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.757870 20268 raft_consensus.cc:374] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.758042 20268 raft_consensus.cc:729] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.758329 20268 consensus_queue.cc:260] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.758582 20268 raft_consensus.cc:388] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.758690 20268 raft_consensus.cc:482] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.758803 20268 raft_consensus.cc:3037] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.760712 20268 raft_consensus.cc:504] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.761018 20268 leader_election.cc:302] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.761224 20268 leader_election.cc:288] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.761312 20273 raft_consensus.cc:2781] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.761473 20273 raft_consensus.cc:686] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.761751 20273 consensus_queue.cc:237] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.762705 20268 ts_tablet_manager.cc:1428] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 02:58:19.766273 19968 catalog_manager.cc:5495] T 4b1d1031995f4f8d901b51e896034647 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.820883 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59254:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.833943 20165 tablet_service.cc:1467] Processing CreateTablet for tablet f3acdf973b9f4008a4b389230226b672 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.27 [id=d880d257e54c45369378e5054e807bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.834506 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3acdf973b9f4008a4b389230226b672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.839082 20268 tablet_bootstrap.cc:492] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.841025 20268 tablet_bootstrap.cc:654] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.843050 20268 tablet_bootstrap.cc:492] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.843227 20268 ts_tablet_manager.cc:1397] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:19.843907 20268 raft_consensus.cc:348] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.844174 20268 raft_consensus.cc:374] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.844267 20268 raft_consensus.cc:729] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.844523 20268 consensus_queue.cc:260] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.844745 20268 raft_consensus.cc:388] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.844867 20268 raft_consensus.cc:482] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.844985 20268 raft_consensus.cc:3037] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.847003 20268 raft_consensus.cc:504] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.847259 20268 leader_election.cc:302] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.847484 20268 leader_election.cc:288] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.847630 20273 raft_consensus.cc:2781] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.847900 20273 raft_consensus.cc:686] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.848297 20268 ts_tablet_manager.cc:1428] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260811 02:58:19.848202 20273 consensus_queue.cc:237] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.852937 19968 catalog_manager.cc:5495] T f3acdf973b9f4008a4b389230226b672 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.861989 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260811 02:58:19.872699 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.007s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":75,"dirs.run_cpu_time_us":250,"dirs.run_wall_time_us":1298,"drs_written":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4,"lbm_write_time_us":625,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:19.873497 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 562 bytes on disk
I20260811 02:58:19.874300 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":134,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.874945 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.305014
I20260811 02:58:19.900861 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.020s sys 0.004s Metrics: {"bytes_written":17957,"cfile_cache_hit":13,"cfile_cache_hit_bytes":11513,"cfile_cache_miss":31,"cfile_cache_miss_bytes":17929,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":2197,"dirs.run_cpu_time_us":656,"dirs.run_wall_time_us":3446,"drs_written":1,"lbm_read_time_us":1088,"lbm_reads_lt_1ms":63,"lbm_write_time_us":856,"lbm_writes_lt_1ms":27,"mutex_wait_us":1337,"num_input_rowsets":2,"peak_mem_usage":7672,"rows_written":57}
I20260811 02:58:19.902113 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2672 bytes on disk
I20260811 02:58:19.902781 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 02:58:19.906029 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59268:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:19.920481 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 0869f35021df410ba0eea471db03fe32 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.28 [id=48dd5ac105f34713ab84a2f1ab83a103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:19.920985 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0869f35021df410ba0eea471db03fe32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:19.926074 20268 tablet_bootstrap.cc:492] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:19.927482 20268 tablet_bootstrap.cc:654] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:19.929589 20268 tablet_bootstrap.cc:492] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:19.929798 20268 ts_tablet_manager.cc:1397] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:19.930454 20268 raft_consensus.cc:348] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.930642 20268 raft_consensus.cc:374] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:19.930740 20268 raft_consensus.cc:729] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:19.931011 20268 consensus_queue.cc:260] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.931208 20268 raft_consensus.cc:388] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:19.931298 20268 raft_consensus.cc:482] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:19.931422 20268 raft_consensus.cc:3037] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:19.933607 20268 raft_consensus.cc:504] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.933923 20268 leader_election.cc:302] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:19.934154 20268 leader_election.cc:288] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:19.934238 20273 raft_consensus.cc:2781] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:19.934476 20273 raft_consensus.cc:686] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:19.934759 20268 ts_tablet_manager.cc:1428] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:19.934787 20273 consensus_queue.cc:237] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:19.938602 19968 catalog_manager.cc:5495] T 0869f35021df410ba0eea471db03fe32 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:19.953440 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:19.965710 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.012s sys 0.000s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":100,"dirs.run_cpu_time_us":262,"dirs.run_wall_time_us":1444,"drs_written":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4,"lbm_write_time_us":929,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:19.966683 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.301776
I20260811 02:58:19.967371 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260811 02:58:19.981683 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.005s sys 0.008s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":79,"dirs.run_cpu_time_us":241,"dirs.run_wall_time_us":1442,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1006,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:19.983119 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.301776
I20260811 02:58:20.000123 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.023s sys 0.004s Metrics: {"bytes_written":18389,"cfile_cache_hit":13,"cfile_cache_hit_bytes":11888,"cfile_cache_miss":31,"cfile_cache_miss_bytes":18556,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":697,"dirs.run_cpu_time_us":666,"dirs.run_wall_time_us":3506,"drs_written":1,"lbm_read_time_us":3022,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":66,"lbm_write_time_us":1232,"lbm_writes_lt_1ms":27,"mutex_wait_us":194,"num_input_rowsets":2,"peak_mem_usage":8132,"rows_written":59,"spinlock_wait_cycles":1792}
I20260811 02:58:20.000893 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59278:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.001204 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2761 bytes on disk
I20260811 02:58:20.001909 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.013604 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.019s sys 0.007s Metrics: {"bytes_written":18389,"cfile_cache_hit":13,"cfile_cache_hit_bytes":11888,"cfile_cache_miss":31,"cfile_cache_miss_bytes":18556,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":726,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":1088,"drs_written":1,"lbm_read_time_us":1364,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1095,"lbm_writes_lt_1ms":27,"mutex_wait_us":142,"num_input_rowsets":2,"peak_mem_usage":8132,"rows_written":59}
I20260811 02:58:20.014773 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2761 bytes on disk
I20260811 02:58:20.015515 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.016288 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260811 02:58:20.016369 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 8192741213224277acaaaac15cc14cee (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.29 [id=ab6ed160723c4d6caa6a66757ffcf854]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.016862 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8192741213224277acaaaac15cc14cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.022249 20268 tablet_bootstrap.cc:492] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.023994 20268 tablet_bootstrap.cc:654] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.026006 20268 tablet_bootstrap.cc:492] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.026209 20268 ts_tablet_manager.cc:1397] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:20.027094 20268 raft_consensus.cc:348] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.027275 20268 raft_consensus.cc:374] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.027379 20268 raft_consensus.cc:729] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.027618 20268 consensus_queue.cc:260] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.027778 20268 raft_consensus.cc:388] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.027923 20268 raft_consensus.cc:482] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.028057 20268 raft_consensus.cc:3037] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.028443 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.010s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":45,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1351,"drs_written":1,"lbm_read_time_us":51,"lbm_reads_lt_1ms":4,"lbm_write_time_us":682,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:20.029333 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.199055
I20260811 02:58:20.030220 20268 raft_consensus.cc:504] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.030494 20268 leader_election.cc:302] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.030754 20268 leader_election.cc:288] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.030815 20273 raft_consensus.cc:2781] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.031586 20273 raft_consensus.cc:686] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.031837 20273 consensus_queue.cc:237] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.036800 19968 catalog_manager.cc:5495] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.037910 20268 ts_tablet_manager.cc:1428] T 8192741213224277acaaaac15cc14cee P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260811 02:58:20.053785 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.021s sys 0.003s Metrics: {"bytes_written":18791,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12223,"cfile_cache_miss":28,"cfile_cache_miss_bytes":18357,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1128,"dirs.run_cpu_time_us":205,"dirs.run_wall_time_us":1493,"drs_written":1,"lbm_read_time_us":1295,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1144,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":9053,"rows_written":61}
I20260811 02:58:20.055256 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2829 bytes on disk
I20260811 02:58:20.057632 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.058395 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260811 02:58:20.063900 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"bytes_written":390,"delete_count":0,"lbm_write_time_us":190,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260811 02:58:20.089155 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59290:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.102901 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 63ebf3c0ae5f4843875c68724bfd578c (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.30 [id=7670b1b36bca4cb18bc61e1671baa566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.103384 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63ebf3c0ae5f4843875c68724bfd578c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.107892 20268 tablet_bootstrap.cc:492] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.109269 20268 tablet_bootstrap.cc:654] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.111596 20268 tablet_bootstrap.cc:492] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.111771 20268 ts_tablet_manager.cc:1397] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260811 02:58:20.112373 20268 raft_consensus.cc:348] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.112572 20268 raft_consensus.cc:374] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.112692 20268 raft_consensus.cc:729] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.112927 20268 consensus_queue.cc:260] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.113130 20268 raft_consensus.cc:388] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.113234 20268 raft_consensus.cc:482] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.113343 20268 raft_consensus.cc:3037] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.115664 20268 raft_consensus.cc:504] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.115969 20268 leader_election.cc:302] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.116173 20268 leader_election.cc:288] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.116288 20273 raft_consensus.cc:2781] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.116492 20273 raft_consensus.cc:686] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.116701 20268 ts_tablet_manager.cc:1428] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:20.116755 20273 consensus_queue.cc:237] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.120527 19968 catalog_manager.cc:5495] T 63ebf3c0ae5f4843875c68724bfd578c P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.154222 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260811 02:58:20.166153 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59292:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.168887 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.009s sys 0.004s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":40,"dirs.run_cpu_time_us":212,"dirs.run_wall_time_us":1437,"drs_written":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1413,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:20.169904 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 554 bytes on disk
I20260811 02:58:20.171357 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":137,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.172032 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.273270
I20260811 02:58:20.180632 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 88f4e144baa644539684454f3d4e2a81 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.31 [id=a2ca67e7cf3b4e70b3b1e0337885540a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.181200 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88f4e144baa644539684454f3d4e2a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.186223 20268 tablet_bootstrap.cc:492] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.187674 20268 tablet_bootstrap.cc:654] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.190163 20268 tablet_bootstrap.cc:492] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.190344 20268 ts_tablet_manager.cc:1397] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:20.191077 20268 raft_consensus.cc:348] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.191253 20268 raft_consensus.cc:374] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.191355 20268 raft_consensus.cc:729] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.191605 20268 consensus_queue.cc:260] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.191777 20268 raft_consensus.cc:388] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.191893 20268 raft_consensus.cc:482] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.191999 20268 raft_consensus.cc:3037] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.194370 20268 raft_consensus.cc:504] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.194649 20268 leader_election.cc:302] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.194860 20268 leader_election.cc:288] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.194957 20273 raft_consensus.cc:2781] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.195188 20273 raft_consensus.cc:686] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.195391 20268 ts_tablet_manager.cc:1428] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:20.195451 20273 consensus_queue.cc:237] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.201607 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.018s sys 0.004s Metrics: {"bytes_written":19259,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12631,"cfile_cache_miss":28,"cfile_cache_miss_bytes":19221,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":988,"dirs.run_cpu_time_us":1076,"dirs.run_wall_time_us":6936,"drs_written":1,"lbm_read_time_us":1177,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1042,"lbm_writes_lt_1ms":27,"mutex_wait_us":58,"num_input_rowsets":2,"peak_mem_usage":8797,"rows_written":63,"spinlock_wait_cycles":6784,"thread_start_us":954,"threads_started":2}
I20260811 02:58:20.202786 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260811 02:58:20.211400 19968 catalog_manager.cc:5495] T 88f4e144baa644539684454f3d4e2a81 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.216641 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.008s sys 0.003s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":76,"dirs.run_cpu_time_us":198,"dirs.run_wall_time_us":4298,"drs_written":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1179,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:20.217787 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.196531
I20260811 02:58:20.245592 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.022s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12963,"cfile_cache_miss":28,"cfile_cache_miss_bytes":19465,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":621,"dirs.run_cpu_time_us":213,"dirs.run_wall_time_us":1334,"drs_written":1,"lbm_read_time_us":1297,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1080,"lbm_writes_lt_1ms":31,"mutex_wait_us":104,"num_input_rowsets":2,"peak_mem_usage":9775,"reinsert_count":0,"rows_written":65,"update_count":1}
I20260811 02:58:20.246717 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3014 bytes on disk
I20260811 02:58:20.247887 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":114,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.252962 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:20.261279 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59296:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.270063 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.012s sys 0.000s Metrics: {"bytes_written":6067,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":263,"dirs.run_wall_time_us":1288,"drs_written":1,"lbm_read_time_us":52,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1010,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:20.271057 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 554 bytes on disk
I20260811 02:58:20.271781 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.273226 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.269074
I20260811 02:58:20.275295 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 5246159a052245049ae7759ce9b6654d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.32 [id=5b55d8017e894e7b95417f6b0402b89a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.275812 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5246159a052245049ae7759ce9b6654d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.282545 20268 tablet_bootstrap.cc:492] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.284312 20268 tablet_bootstrap.cc:654] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.286710 20268 tablet_bootstrap.cc:492] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.286897 20268 ts_tablet_manager.cc:1397] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:20.287729 20268 raft_consensus.cc:348] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.287938 20268 raft_consensus.cc:374] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.288059 20268 raft_consensus.cc:729] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.288296 20268 consensus_queue.cc:260] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.288551 20268 raft_consensus.cc:388] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.288651 20268 raft_consensus.cc:482] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.288776 20268 raft_consensus.cc:3037] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.291531 20268 raft_consensus.cc:504] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.291828 20268 leader_election.cc:302] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.292050 20268 leader_election.cc:288] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.292176 20273 raft_consensus.cc:2781] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.292346 20273 raft_consensus.cc:686] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.292680 20268 ts_tablet_manager.cc:1428] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260811 02:58:20.292645 20273 consensus_queue.cc:237] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.297194 19970 catalog_manager.cc:5495] T 5246159a052245049ae7759ce9b6654d P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.299113 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.021s sys 0.004s Metrics: {"bytes_written":19710,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12996,"cfile_cache_miss":28,"cfile_cache_miss_bytes":19764,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":838,"dirs.run_cpu_time_us":215,"dirs.run_wall_time_us":1384,"drs_written":1,"lbm_read_time_us":1193,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1164,"lbm_writes_lt_1ms":27,"mutex_wait_us":28,"num_input_rowsets":2,"peak_mem_usage":9053,"rows_written":65,"spinlock_wait_cycles":4992}
I20260811 02:58:20.300037 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000008
I20260811 02:58:20.314994 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:20.325886 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.013s sys 0.001s Metrics: {"bytes_written":335,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":75,"dirs.run_cpu_time_us":200,"dirs.run_wall_time_us":1305,"drs_written":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1193,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:20.326855 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3265 bytes on disk
I20260811 02:58:20.327778 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":163,"lbm_reads_lt_1ms":8}
I20260811 02:58:20.328516 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.463347
I20260811 02:58:20.329227 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.009s sys 0.003s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":78,"dirs.run_cpu_time_us":178,"dirs.run_wall_time_us":1227,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":983,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:20.330121 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.265433
I20260811 02:58:20.338925 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":579,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":146,"lbm_read_time_us":442,"lbm_reads_lt_1ms":21,"lbm_write_time_us":346,"lbm_writes_lt_1ms":9,"mutex_wait_us":40,"reinsert_count":0,"update_count":1}
I20260811 02:58:20.339563 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.238294
I20260811 02:58:20.371227 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59308:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.372246 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.042s user 0.023s sys 0.004s Metrics: {"bytes_written":20138,"cfile_cache_hit":13,"cfile_cache_hit_bytes":13368,"cfile_cache_miss":31,"cfile_cache_miss_bytes":20718,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":710,"dirs.run_cpu_time_us":830,"dirs.run_wall_time_us":4163,"drs_written":1,"lbm_read_time_us":1364,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1053,"lbm_writes_lt_1ms":27,"mutex_wait_us":579,"num_input_rowsets":2,"peak_mem_usage":9566,"rows_written":67}
I20260811 02:58:20.373659 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3112 bytes on disk
I20260811 02:58:20.374554 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":130,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.375672 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.024s sys 0.008s Metrics: {"bytes_written":20138,"cfile_cache_hit":13,"cfile_cache_hit_bytes":13409,"cfile_cache_miss":31,"cfile_cache_miss_bytes":20477,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1566,"dirs.run_cpu_time_us":1060,"dirs.run_wall_time_us":7197,"drs_written":1,"lbm_read_time_us":1332,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1215,"lbm_writes_lt_1ms":27,"mutex_wait_us":179,"num_input_rowsets":2,"peak_mem_usage":10031,"rows_written":67,"thread_start_us":7211,"threads_started":1}
I20260811 02:58:20.376521 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3112 bytes on disk
I20260811 02:58:20.378042 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.387737 20165 tablet_service.cc:1467] Processing CreateTablet for tablet d9dbb45474924b538fb76c5b0afa31a8 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.33 [id=107102e8ec4f4fbb90c71c5929df668b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.388340 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9dbb45474924b538fb76c5b0afa31a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.393821 20268 tablet_bootstrap.cc:492] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.395555 20268 tablet_bootstrap.cc:654] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.398025 20268 tablet_bootstrap.cc:492] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.398234 20268 ts_tablet_manager.cc:1397] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:20.398923 20268 raft_consensus.cc:348] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.399174 20268 raft_consensus.cc:374] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.399300 20268 raft_consensus.cc:729] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.399572 20268 consensus_queue.cc:260] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.399766 20268 raft_consensus.cc:388] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.399893 20268 raft_consensus.cc:482] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.400020 20268 raft_consensus.cc:3037] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.402701 20268 raft_consensus.cc:504] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.402971 20268 leader_election.cc:302] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.403191 20268 leader_election.cc:288] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.403254 20273 raft_consensus.cc:2781] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.403458 20273 raft_consensus.cc:686] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.403705 20268 ts_tablet_manager.cc:1428] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260811 02:58:20.403712 20273 consensus_queue.cc:237] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.408473 19968 catalog_manager.cc:5495] T d9dbb45474924b538fb76c5b0afa31a8 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.445852 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59310:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.459210 20165 tablet_service.cc:1467] Processing CreateTablet for tablet aefde8f78bb241c7a5f16d3d3fba6148 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.34 [id=eedc80567c9b4332ba797f35ba32b688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.459729 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aefde8f78bb241c7a5f16d3d3fba6148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.464486 20268 tablet_bootstrap.cc:492] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.465989 20268 tablet_bootstrap.cc:654] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.467998 20268 tablet_bootstrap.cc:492] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.468178 20268 ts_tablet_manager.cc:1397] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:20.468818 20268 raft_consensus.cc:348] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.469023 20268 raft_consensus.cc:374] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.469147 20268 raft_consensus.cc:729] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.469391 20268 consensus_queue.cc:260] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.469659 20268 raft_consensus.cc:388] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.469758 20268 raft_consensus.cc:482] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.469870 20268 raft_consensus.cc:3037] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.471961 20268 raft_consensus.cc:504] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.472226 20268 leader_election.cc:302] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.472424 20268 leader_election.cc:288] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.472560 20273 raft_consensus.cc:2781] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.472738 20273 raft_consensus.cc:686] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.472982 20273 consensus_queue.cc:237] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.473225 20268 ts_tablet_manager.cc:1428] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260811 02:58:20.476871 19968 catalog_manager.cc:5495] T aefde8f78bb241c7a5f16d3d3fba6148 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.498982 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260811 02:58:20.514810 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.011s sys 0.003s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":71,"dirs.run_cpu_time_us":211,"dirs.run_wall_time_us":1467,"drs_written":1,"lbm_read_time_us":139,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1023,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:20.515934 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.288757
I20260811 02:58:20.523298 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59326:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.537573 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 151c21317a53405fa6cc17abe39e5735 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.35 [id=f9ec10c70d7d4496bb859c8c62ba33f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.538118 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 151c21317a53405fa6cc17abe39e5735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.543017 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.019s sys 0.004s Metrics: {"bytes_written":21019,"cfile_cache_hit":13,"cfile_cache_hit_bytes":14108,"cfile_cache_miss":31,"cfile_cache_miss_bytes":21732,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":520,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":2655,"drs_written":1,"lbm_read_time_us":1064,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1064,"lbm_writes_lt_1ms":27,"mutex_wait_us":55,"num_input_rowsets":2,"peak_mem_usage":10079,"rows_written":71,"spinlock_wait_cycles":10112}
I20260811 02:58:20.543910 20268 tablet_bootstrap.cc:492] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.544234 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000008
I20260811 02:58:20.545509 20268 tablet_bootstrap.cc:654] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.547683 20268 tablet_bootstrap.cc:492] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.547873 20268 ts_tablet_manager.cc:1397] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:20.548521 20268 raft_consensus.cc:348] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.548702 20268 raft_consensus.cc:374] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.548810 20268 raft_consensus.cc:729] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.549084 20268 consensus_queue.cc:260] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.549335 20268 raft_consensus.cc:388] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.549451 20268 raft_consensus.cc:482] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.549578 20268 raft_consensus.cc:3037] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.551993 20268 raft_consensus.cc:504] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.552284 20268 leader_election.cc:302] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.552520 20268 leader_election.cc:288] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.552604 20273 raft_consensus.cc:2781] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.552794 20273 raft_consensus.cc:686] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.553061 20273 consensus_queue.cc:237] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.553380 20268 ts_tablet_manager.cc:1428] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260811 02:58:20.555892 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.005s sys 0.004s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":101,"dirs.run_cpu_time_us":202,"dirs.run_wall_time_us":1510,"drs_written":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4,"lbm_write_time_us":771,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:20.556905 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.184758
I20260811 02:58:20.557086 19968 catalog_manager.cc:5495] T 151c21317a53405fa6cc17abe39e5735 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.585359 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.017s sys 0.008s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":14443,"cfile_cache_miss":28,"cfile_cache_miss_bytes":21617,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":725,"dirs.run_cpu_time_us":162,"dirs.run_wall_time_us":1018,"drs_written":1,"lbm_read_time_us":1201,"lbm_reads_lt_1ms":64,"lbm_write_time_us":3439,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":29,"mutex_wait_us":59,"num_input_rowsets":2,"peak_mem_usage":10799,"reinsert_count":0,"rows_written":73,"spinlock_wait_cycles":2816,"update_count":1}
I20260811 02:58:20.586294 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3342 bytes on disk
I20260811 02:58:20.587074 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":113,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.611419 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59332:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.625384 20165 tablet_service.cc:1467] Processing CreateTablet for tablet b25bd74c648b4ac8b247dbc2b0508b30 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.36 [id=691f1b8fcf30480c806ef759c9a5bb11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.625669 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260811 02:58:20.625958 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b25bd74c648b4ac8b247dbc2b0508b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.629032 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260811 02:58:20.632503 20268 tablet_bootstrap.cc:492] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.634642 20268 tablet_bootstrap.cc:654] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.638047 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.011s sys 0.001s Metrics: {"bytes_written":6463,"cfile_init":1,"dirs.queue_time_us":102,"dirs.run_cpu_time_us":252,"dirs.run_wall_time_us":1400,"drs_written":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1015,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260811 02:58:20.639194 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 588 bytes on disk
I20260811 02:58:20.639829 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.640597 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.303781
I20260811 02:58:20.640055 20268 tablet_bootstrap.cc:492] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.640913 20268 ts_tablet_manager.cc:1397] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 02:58:20.641868 20268 raft_consensus.cc:348] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.642171 20268 raft_consensus.cc:374] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.642319 20268 raft_consensus.cc:729] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.642467 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6463,"cfile_init":1,"dirs.queue_time_us":97,"dirs.run_cpu_time_us":239,"dirs.run_wall_time_us":1280,"drs_written":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4,"lbm_write_time_us":999,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260811 02:58:20.642585 20268 consensus_queue.cc:260] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.642828 20268 raft_consensus.cc:388] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.642917 20268 raft_consensus.cc:482] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.643014 20268 raft_consensus.cc:3037] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.643431 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.303781
I20260811 02:58:20.645920 20268 raft_consensus.cc:504] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.646255 20268 leader_election.cc:302] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.646627 20273 raft_consensus.cc:2781] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.646819 20273 raft_consensus.cc:686] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.647087 20273 consensus_queue.cc:237] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.651614 20268 leader_election.cc:288] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.651988 19968 catalog_manager.cc:5495] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.652550 20268 ts_tablet_manager.cc:1428] T b25bd74c648b4ac8b247dbc2b0508b30 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.012s user 0.001s sys 0.005s
I20260811 02:58:20.671558 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.022s sys 0.003s Metrics: {"bytes_written":334,"cfile_cache_hit":12,"cfile_cache_hit_bytes":14802,"cfile_cache_miss":28,"cfile_cache_miss_bytes":22344,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":835,"dirs.run_cpu_time_us":692,"dirs.run_wall_time_us":5536,"drs_written":1,"lbm_read_time_us":1060,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1294,"lbm_writes_lt_1ms":31,"mutex_wait_us":51,"num_input_rowsets":2,"peak_mem_usage":10287,"reinsert_count":0,"rows_written":75,"spinlock_wait_cycles":2304,"update_count":1}
I20260811 02:58:20.672951 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3425 bytes on disk
I20260811 02:58:20.673764 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.683655 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.026s sys 0.000s Metrics: {"bytes_written":334,"cfile_cache_hit":12,"cfile_cache_hit_bytes":14802,"cfile_cache_miss":28,"cfile_cache_miss_bytes":22344,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":775,"dirs.run_cpu_time_us":181,"dirs.run_wall_time_us":2456,"drs_written":1,"lbm_read_time_us":1223,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1187,"lbm_writes_lt_1ms":31,"mutex_wait_us":25,"num_input_rowsets":2,"peak_mem_usage":10287,"reinsert_count":0,"rows_written":75,"spinlock_wait_cycles":31104,"update_count":1}
I20260811 02:58:20.684679 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260811 02:58:20.689298 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.001s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":125,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:20.689857 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3425 bytes on disk
I20260811 02:58:20.690619 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.691273 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260811 02:58:20.699095 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":72,"lbm_read_time_us":232,"lbm_reads_lt_1ms":12,"lbm_write_time_us":187,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260811 02:58:20.716912 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59338:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.729533 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 46e37ea322d241e7aff59bfa567a5999 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.37 [id=2265222b7e3449948331f4413fb4f9e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.730079 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e37ea322d241e7aff59bfa567a5999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.736430 20268 tablet_bootstrap.cc:492] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.738170 20268 tablet_bootstrap.cc:654] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.740674 20268 tablet_bootstrap.cc:492] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.740850 20268 ts_tablet_manager.cc:1397] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:20.741663 20268 raft_consensus.cc:348] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.741922 20268 raft_consensus.cc:374] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.742003 20268 raft_consensus.cc:729] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.742226 20268 consensus_queue.cc:260] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.742411 20268 raft_consensus.cc:388] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.742503 20268 raft_consensus.cc:482] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.742600 20268 raft_consensus.cc:3037] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.745505 20268 raft_consensus.cc:504] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.745818 20268 leader_election.cc:302] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.746054 20268 leader_election.cc:288] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.746137 20273 raft_consensus.cc:2781] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.746342 20273 raft_consensus.cc:686] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.746528 20268 ts_tablet_manager.cc:1428] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:58:20.746606 20273 consensus_queue.cc:237] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.753110 19970 catalog_manager.cc:5495] T 46e37ea322d241e7aff59bfa567a5999 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.796931 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59350:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.809916 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 181abca1accb4a69b49345f856fcd760 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.38 [id=1305b950ab4a489abdc80a13c9b0cbfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.810415 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 181abca1accb4a69b49345f856fcd760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.815256 20268 tablet_bootstrap.cc:492] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.816776 20268 tablet_bootstrap.cc:654] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.818919 20268 tablet_bootstrap.cc:492] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.819101 20268 ts_tablet_manager.cc:1397] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:20.819777 20268 raft_consensus.cc:348] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.819990 20268 raft_consensus.cc:374] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.820114 20268 raft_consensus.cc:729] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.820362 20268 consensus_queue.cc:260] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.820596 20268 raft_consensus.cc:388] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.820703 20268 raft_consensus.cc:482] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.820814 20268 raft_consensus.cc:3037] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.822992 20268 raft_consensus.cc:504] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.823297 20268 leader_election.cc:302] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.823521 20268 leader_election.cc:288] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.823679 20273 raft_consensus.cc:2781] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.823863 20273 raft_consensus.cc:686] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.824106 20268 ts_tablet_manager.cc:1428] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:20.824132 20273 consensus_queue.cc:237] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.830708 19970 catalog_manager.cc:5495] T 181abca1accb4a69b49345f856fcd760 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.838116 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260811 02:58:20.849896 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.007s sys 0.004s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":71,"dirs.run_cpu_time_us":195,"dirs.run_wall_time_us":1382,"drs_written":1,"lbm_read_time_us":57,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1004,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:20.850769 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 549 bytes on disk
I20260811 02:58:20.851464 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.852227 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.280334
I20260811 02:58:20.876971 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59356:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.878976 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.018s sys 0.008s Metrics: {"bytes_written":22770,"cfile_cache_hit":13,"cfile_cache_hit_bytes":15588,"cfile_cache_miss":31,"cfile_cache_miss_bytes":23942,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":765,"dirs.run_cpu_time_us":284,"dirs.run_wall_time_us":1244,"drs_written":1,"lbm_read_time_us":1332,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1174,"lbm_writes_lt_1ms":27,"mutex_wait_us":136,"num_input_rowsets":2,"peak_mem_usage":11103,"rows_written":79,"spinlock_wait_cycles":11648}
I20260811 02:58:20.880121 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3614 bytes on disk
I20260811 02:58:20.880909 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.891353 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 1d6d353ab27241b5a58e97713c1ad8e6 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.39 [id=d8722a1aea23476f89386bb7a1ef2e91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.891888 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d6d353ab27241b5a58e97713c1ad8e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.897312 20268 tablet_bootstrap.cc:492] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.898931 20268 tablet_bootstrap.cc:654] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.901001 20268 tablet_bootstrap.cc:492] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.901198 20268 ts_tablet_manager.cc:1397] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:20.901932 20268 raft_consensus.cc:348] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.902146 20268 raft_consensus.cc:374] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.902254 20268 raft_consensus.cc:729] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.902508 20268 consensus_queue.cc:260] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.902695 20268 raft_consensus.cc:388] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.902814 20268 raft_consensus.cc:482] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.902933 20268 raft_consensus.cc:3037] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.904839 20268 raft_consensus.cc:504] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.905110 20268 leader_election.cc:302] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.905339 20268 leader_election.cc:288] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.905454 20273 raft_consensus.cc:2781] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.905743 20273 raft_consensus.cc:686] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.905891 20268 ts_tablet_manager.cc:1428] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:20.906038 20273 consensus_queue.cc:237] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.909865 19970 catalog_manager.cc:5495] T 1d6d353ab27241b5a58e97713c1ad8e6 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:20.924769 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:20.937502 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.011s sys 0.000s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":70,"dirs.run_cpu_time_us":248,"dirs.run_wall_time_us":1274,"drs_written":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4,"lbm_write_time_us":739,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:20.939405 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.252372
I20260811 02:58:20.950268 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260811 02:58:20.951808 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59362:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:20.968138 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 7d0d524e96844025988a7dc798b367c0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.40 [id=5855c365f1714d47a7a108c49934e9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:20.968649 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d0d524e96844025988a7dc798b367c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:20.969514 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.008s sys 0.002s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":231,"dirs.run_cpu_time_us":208,"dirs.run_wall_time_us":1589,"drs_written":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4,"lbm_write_time_us":885,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6,"spinlock_wait_cycles":33536}
I20260811 02:58:20.970988 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 518 bytes on disk
I20260811 02:58:20.971670 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260811 02:58:20.972384 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.252126
I20260811 02:58:20.974581 20268 tablet_bootstrap.cc:492] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:20.976236 20268 tablet_bootstrap.cc:654] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:20.982848 20268 tablet_bootstrap.cc:492] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:20.983052 20268 ts_tablet_manager.cc:1397] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 02:58:20.983830 20268 raft_consensus.cc:348] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.984043 20268 raft_consensus.cc:374] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:20.984145 20268 raft_consensus.cc:729] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:20.984457 20268 consensus_queue.cc:260] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.984689 20268 raft_consensus.cc:388] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:20.984781 20268 raft_consensus.cc:482] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:20.984899 20268 raft_consensus.cc:3037] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:20.987506 20268 raft_consensus.cc:504] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.987823 20268 leader_election.cc:302] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:20.988039 20268 leader_election.cc:288] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:20.988129 20273 raft_consensus.cc:2781] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:20.988263 20273 raft_consensus.cc:686] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:20.988649 20268 ts_tablet_manager.cc:1428] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:20.988552 20273 consensus_queue.cc:237] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:20.989753 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.049s user 0.034s sys 0.006s Metrics: {"bytes_written":23213,"cfile_cache_hit":13,"cfile_cache_hit_bytes":15953,"cfile_cache_miss":31,"cfile_cache_miss_bytes":24399,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1538,"dirs.run_cpu_time_us":1088,"dirs.run_wall_time_us":8002,"drs_written":1,"lbm_read_time_us":4571,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":62,"lbm_write_time_us":1763,"lbm_writes_lt_1ms":27,"mutex_wait_us":999,"num_input_rowsets":2,"peak_mem_usage":11356,"rows_written":81,"spinlock_wait_cycles":896,"thread_start_us":13044,"threads_started":1}
I20260811 02:58:20.990821 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000015
I20260811 02:58:20.997354 19970 catalog_manager.cc:5495] T 7d0d524e96844025988a7dc798b367c0 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.004487 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.011s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":41,"dirs.run_cpu_time_us":185,"dirs.run_wall_time_us":1302,"drs_written":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1051,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":1280}
I20260811 02:58:21.005520 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3928 bytes on disk
I20260811 02:58:21.006405 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":159,"lbm_reads_lt_1ms":8}
I20260811 02:58:21.007157 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.213440
I20260811 02:58:21.013643 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.028s sys 0.002s Metrics: {"bytes_written":23213,"cfile_cache_hit":13,"cfile_cache_hit_bytes":15953,"cfile_cache_miss":31,"cfile_cache_miss_bytes":24511,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":3106,"dirs.run_cpu_time_us":813,"dirs.run_wall_time_us":3789,"drs_written":1,"lbm_read_time_us":2115,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1374,"lbm_writes_lt_1ms":27,"mutex_wait_us":2276,"num_input_rowsets":2,"peak_mem_usage":11563,"rows_written":81}
I20260811 02:58:21.014966 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260811 02:58:21.027920 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.005s sys 0.005s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":72,"dirs.run_cpu_time_us":250,"dirs.run_wall_time_us":1271,"drs_written":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4,"lbm_write_time_us":950,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:21.030777 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4052 bytes on disk
I20260811 02:58:21.031658 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":173,"lbm_reads_lt_1ms":8}
I20260811 02:58:21.032508 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.215189
I20260811 02:58:21.052300 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.020s sys 0.019s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":16298,"cfile_cache_miss":28,"cfile_cache_miss_bytes":24322,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":2529,"dirs.run_cpu_time_us":1299,"dirs.run_wall_time_us":6253,"drs_written":1,"lbm_read_time_us":1122,"lbm_reads_lt_1ms":56,"lbm_write_time_us":971,"lbm_writes_lt_1ms":31,"mutex_wait_us":58,"num_input_rowsets":2,"peak_mem_usage":12284,"reinsert_count":0,"rows_written":83,"spinlock_wait_cycles":4352,"thread_start_us":14983,"threads_started":1,"update_count":1}
I20260811 02:58:21.054122 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3763 bytes on disk
I20260811 02:58:21.059897 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":109,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.062613 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59370:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.066241 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.029s sys 0.000s Metrics: {"bytes_written":23642,"cfile_cache_hit":12,"cfile_cache_hit_bytes":16336,"cfile_cache_miss":28,"cfile_cache_miss_bytes":24604,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":3790,"dirs.run_cpu_time_us":527,"dirs.run_wall_time_us":3254,"drs_written":1,"lbm_read_time_us":1767,"lbm_reads_lt_1ms":56,"lbm_write_time_us":2094,"lbm_writes_lt_1ms":27,"mutex_wait_us":666,"num_input_rowsets":2,"peak_mem_usage":12284,"rows_written":83,"spinlock_wait_cycles":2304}
I20260811 02:58:21.067183 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3775 bytes on disk
I20260811 02:58:21.068322 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.076895 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 352551b8e8ef4f1ca9f8463540ee3a80 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.41 [id=c12740aa569c43b087ef0d88dda0348c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.077366 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 352551b8e8ef4f1ca9f8463540ee3a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.082103 20268 tablet_bootstrap.cc:492] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.083629 20268 tablet_bootstrap.cc:654] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.085848 20268 tablet_bootstrap.cc:492] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.085999 20268 ts_tablet_manager.cc:1397] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:21.086670 20268 raft_consensus.cc:348] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.086933 20268 raft_consensus.cc:374] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.087049 20268 raft_consensus.cc:729] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.087306 20268 consensus_queue.cc:260] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.087491 20268 raft_consensus.cc:388] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.087620 20268 raft_consensus.cc:482] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.087731 20268 raft_consensus.cc:3037] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.089886 20268 raft_consensus.cc:504] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.090090 20268 leader_election.cc:302] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.090298 20268 leader_election.cc:288] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.090399 20271 raft_consensus.cc:2781] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.090564 20271 raft_consensus.cc:686] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.090871 20268 ts_tablet_manager.cc:1428] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260811 02:58:21.090787 20271 consensus_queue.cc:237] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.097002 19970 catalog_manager.cc:5495] T 352551b8e8ef4f1ca9f8463540ee3a80 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.132198 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260811 02:58:21.140184 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59378:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.144933 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.012s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":287,"dirs.run_wall_time_us":1364,"drs_written":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4,"lbm_write_time_us":992,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:21.145999 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.249986
I20260811 02:58:21.155627 20165 tablet_service.cc:1467] Processing CreateTablet for tablet b69d3681354645ed88b0545bc13eccb1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.42 [id=09fc9f22d7fe45e5820ebe7c0b01cf39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.156093 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b69d3681354645ed88b0545bc13eccb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.160815 20268 tablet_bootstrap.cc:492] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.162659 20268 tablet_bootstrap.cc:654] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.164512 20268 tablet_bootstrap.cc:492] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.164690 20268 ts_tablet_manager.cc:1397] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:21.165518 20268 raft_consensus.cc:348] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.165807 20268 raft_consensus.cc:374] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.165921 20268 raft_consensus.cc:729] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.166216 20268 consensus_queue.cc:260] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.166433 20268 raft_consensus.cc:388] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.166524 20268 raft_consensus.cc:482] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.166675 20268 raft_consensus.cc:3037] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.168614 20268 raft_consensus.cc:504] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.168897 20268 leader_election.cc:302] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.169121 20268 leader_election.cc:288] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.169231 20271 raft_consensus.cc:2781] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.169416 20271 raft_consensus.cc:686] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.169732 20271 consensus_queue.cc:237] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.169600 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.022s sys 0.000s Metrics: {"bytes_written":24086,"cfile_cache_hit":12,"cfile_cache_hit_bytes":16696,"cfile_cache_miss":28,"cfile_cache_miss_bytes":25130,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1171,"dirs.run_cpu_time_us":209,"dirs.run_wall_time_us":1346,"drs_written":1,"lbm_read_time_us":1116,"lbm_reads_lt_1ms":60,"lbm_write_time_us":851,"lbm_writes_lt_1ms":27,"mutex_wait_us":1,"num_input_rowsets":2,"peak_mem_usage":12028,"rows_written":85}
I20260811 02:58:21.170653 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000007
I20260811 02:58:21.171060 20268 ts_tablet_manager.cc:1428] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260811 02:58:21.178427 19970 catalog_manager.cc:5495] T b69d3681354645ed88b0545bc13eccb1 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.185384 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.009s sys 0.001s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":563,"dirs.run_cpu_time_us":179,"dirs.run_wall_time_us":1096,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1010,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:21.186388 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4092 bytes on disk
I20260811 02:58:21.187239 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":156,"lbm_reads_lt_1ms":8}
I20260811 02:58:21.187897 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.240712
I20260811 02:58:21.217293 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.018s sys 0.008s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":17038,"cfile_cache_miss":28,"cfile_cache_miss_bytes":25392,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":718,"dirs.run_cpu_time_us":228,"dirs.run_wall_time_us":4975,"drs_written":1,"lbm_read_time_us":1160,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1188,"lbm_writes_lt_1ms":31,"mutex_wait_us":140,"num_input_rowsets":2,"peak_mem_usage":12796,"reinsert_count":0,"rows_written":87,"update_count":1}
I20260811 02:58:21.218243 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3936 bytes on disk
I20260811 02:58:21.218927 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.244473 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59390:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.257941 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 51650aa987754d8283aa1f6870cc57e1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.43 [id=f680f490dd854f1fac51ee476be4ad60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.258522 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51650aa987754d8283aa1f6870cc57e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.263808 20268 tablet_bootstrap.cc:492] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.265255 20268 tablet_bootstrap.cc:654] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.267537 20268 tablet_bootstrap.cc:492] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.267750 20268 ts_tablet_manager.cc:1397] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:21.268391 20268 raft_consensus.cc:348] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.268610 20268 raft_consensus.cc:374] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.268712 20268 raft_consensus.cc:729] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.268959 20268 consensus_queue.cc:260] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.269181 20268 raft_consensus.cc:388] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.269301 20268 raft_consensus.cc:482] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.269418 20268 raft_consensus.cc:3037] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.271613 20268 raft_consensus.cc:504] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.271876 20268 leader_election.cc:302] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.272131 20268 leader_election.cc:288] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.272176 20271 raft_consensus.cc:2781] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.272377 20271 raft_consensus.cc:686] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.272629 20271 consensus_queue.cc:237] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.273067 20268 ts_tablet_manager.cc:1428] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260811 02:58:21.278330 19970 catalog_manager.cc:5495] T 51650aa987754d8283aa1f6870cc57e1 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.311190 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260811 02:58:21.319972 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:21.321388 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59402:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.334892 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.005s sys 0.008s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":77,"dirs.run_cpu_time_us":248,"dirs.run_wall_time_us":1373,"drs_written":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4,"lbm_write_time_us":917,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:21.336581 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.013s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":46,"dirs.run_cpu_time_us":200,"dirs.run_wall_time_us":1151,"drs_written":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1449,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:21.337574 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 548 bytes on disk
I20260811 02:58:21.337540 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 43ca9b1fdb2d4eb29280ff9bd0094c59 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.44 [id=47809870b6df478dbba0a7946d72df51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.338174 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43ca9b1fdb2d4eb29280ff9bd0094c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.338235 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.338954 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.274518
I20260811 02:58:21.340638 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.273426
I20260811 02:58:21.344825 20268 tablet_bootstrap.cc:492] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.346741 20268 tablet_bootstrap.cc:654] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.355360 20268 tablet_bootstrap.cc:492] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.355581 20268 ts_tablet_manager.cc:1397] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260811 02:58:21.356524 20268 raft_consensus.cc:348] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.356760 20268 raft_consensus.cc:374] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.356889 20268 raft_consensus.cc:729] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.357194 20268 consensus_queue.cc:260] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.357462 20268 raft_consensus.cc:388] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.357592 20268 raft_consensus.cc:482] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.357769 20268 raft_consensus.cc:3037] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.360714 20268 raft_consensus.cc:504] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.361022 20268 leader_election.cc:302] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.361373 20271 raft_consensus.cc:2781] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.361550 20271 raft_consensus.cc:686] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.362740 20271 consensus_queue.cc:237] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.368533 20268 leader_election.cc:288] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.369681 20268 ts_tablet_manager.cc:1428] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260811 02:58:21.371906 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.024s sys 0.004s Metrics: {"bytes_written":24964,"cfile_cache_hit":13,"cfile_cache_hit_bytes":17433,"cfile_cache_miss":31,"cfile_cache_miss_bytes":26621,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":534,"dirs.run_cpu_time_us":868,"dirs.run_wall_time_us":4472,"drs_written":1,"lbm_read_time_us":1450,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1227,"lbm_writes_lt_1ms":27,"mutex_wait_us":222,"num_input_rowsets":2,"peak_mem_usage":12593,"rows_written":89}
I20260811 02:58:21.373183 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4033 bytes on disk
I20260811 02:58:21.374102 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":157,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.374820 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260811 02:58:21.375376 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.026s sys 0.000s Metrics: {"bytes_written":24964,"cfile_cache_hit":12,"cfile_cache_hit_bytes":17436,"cfile_cache_miss":28,"cfile_cache_miss_bytes":26244,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1246,"dirs.run_cpu_time_us":714,"dirs.run_wall_time_us":4588,"drs_written":1,"lbm_read_time_us":1520,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1055,"lbm_writes_lt_1ms":27,"mutex_wait_us":246,"num_input_rowsets":2,"peak_mem_usage":12540,"rows_written":89}
I20260811 02:58:21.379647 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260811 02:58:21.383028 19968 catalog_manager.cc:5495] T 43ca9b1fdb2d4eb29280ff9bd0094c59 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.393606 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.008s sys 0.003s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":70,"dirs.run_cpu_time_us":195,"dirs.run_wall_time_us":3891,"drs_written":1,"lbm_read_time_us":120,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1007,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:21.394603 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.209212
I20260811 02:58:21.403478 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.009s sys 0.004s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":848,"dirs.run_cpu_time_us":160,"dirs.run_wall_time_us":1460,"drs_written":1,"lbm_read_time_us":126,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1185,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:21.404496 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.459198
I20260811 02:58:21.417375 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.005s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":576,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":155,"lbm_read_time_us":386,"lbm_reads_lt_1ms":21,"lbm_write_time_us":374,"lbm_writes_lt_1ms":9,"reinsert_count":0,"spinlock_wait_cycles":15616,"update_count":1}
I20260811 02:58:21.418155 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4268 bytes on disk
I20260811 02:58:21.418926 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":158,"lbm_reads_lt_1ms":8}
I20260811 02:58:21.419713 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.210697
I20260811 02:58:21.426136 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.014s sys 0.011s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":17778,"cfile_cache_miss":28,"cfile_cache_miss_bytes":26494,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":933,"dirs.run_cpu_time_us":954,"dirs.run_wall_time_us":4708,"drs_written":1,"lbm_read_time_us":1157,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1127,"lbm_writes_lt_1ms":31,"mutex_wait_us":150,"num_input_rowsets":2,"peak_mem_usage":13513,"reinsert_count":0,"rows_written":91,"spinlock_wait_cycles":2304,"update_count":1}
I20260811 02:58:21.427165 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4119 bytes on disk
I20260811 02:58:21.427873 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.449007 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59418:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.451660 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.025s sys 0.000s Metrics: {"bytes_written":25391,"cfile_cache_hit":13,"cfile_cache_hit_bytes":17846,"cfile_cache_miss":31,"cfile_cache_miss_bytes":26952,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1357,"dirs.run_cpu_time_us":1088,"dirs.run_wall_time_us":6369,"drs_written":1,"lbm_read_time_us":1317,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1279,"lbm_writes_lt_1ms":27,"mutex_wait_us":307,"num_input_rowsets":2,"peak_mem_usage":13513,"rows_written":91,"thread_start_us":857,"threads_started":2}
I20260811 02:58:21.452648 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4135 bytes on disk
I20260811 02:58:21.453353 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.464216 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 7a97ed8cc59041f19c32c3cd85cb1a98 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.45 [id=babd0d6225f14106bbf58af9271f61e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.464619 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a97ed8cc59041f19c32c3cd85cb1a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.469878 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:21.469952 20268 tablet_bootstrap.cc:492] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.471658 20268 tablet_bootstrap.cc:654] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.474083 20268 tablet_bootstrap.cc:492] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.474269 20268 ts_tablet_manager.cc:1397] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:21.474963 20268 raft_consensus.cc:348] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.475152 20268 raft_consensus.cc:374] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.475255 20268 raft_consensus.cc:729] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.475529 20268 consensus_queue.cc:260] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.475687 20268 raft_consensus.cc:388] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.475819 20268 raft_consensus.cc:482] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.475934 20268 raft_consensus.cc:3037] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.478158 20268 raft_consensus.cc:504] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.478442 20268 leader_election.cc:302] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.478721 20268 leader_election.cc:288] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.479068 20271 raft_consensus.cc:2781] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.479238 20271 raft_consensus.cc:686] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.479235 20268 ts_tablet_manager.cc:1428] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:21.479514 20271 consensus_queue.cc:237] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.481606 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.010s sys 0.000s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":39,"dirs.run_cpu_time_us":185,"dirs.run_wall_time_us":1286,"drs_written":1,"lbm_read_time_us":116,"lbm_reads_lt_1ms":4,"lbm_write_time_us":799,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:21.482713 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242192
I20260811 02:58:21.484493 19968 catalog_manager.cc:5495] T 7a97ed8cc59041f19c32c3cd85cb1a98 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.513676 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.020s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":18170,"cfile_cache_miss":31,"cfile_cache_miss_bytes":27678,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1100,"dirs.run_cpu_time_us":261,"dirs.run_wall_time_us":1360,"drs_written":1,"lbm_read_time_us":1265,"lbm_reads_lt_1ms":67,"lbm_write_time_us":900,"lbm_writes_lt_1ms":31,"mutex_wait_us":41,"num_input_rowsets":2,"peak_mem_usage":13276,"reinsert_count":0,"rows_written":93,"spinlock_wait_cycles":12416,"update_count":1}
I20260811 02:58:21.515079 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4203 bytes on disk
I20260811 02:58:21.515750 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.527765 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59420:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.541345 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 5c447430652a4bf386711fb8e329ca96 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.46 [id=1096198d7b7f4ea88cfae4cc5921f20f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.541973 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c447430652a4bf386711fb8e329ca96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.547079 20268 tablet_bootstrap.cc:492] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.548655 20268 tablet_bootstrap.cc:654] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.551160 20268 tablet_bootstrap.cc:492] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.551362 20268 ts_tablet_manager.cc:1397] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:21.552062 20268 raft_consensus.cc:348] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.552269 20268 raft_consensus.cc:374] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.552397 20268 raft_consensus.cc:729] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.552640 20268 consensus_queue.cc:260] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.552865 20268 raft_consensus.cc:388] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.552961 20268 raft_consensus.cc:482] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.553107 20268 raft_consensus.cc:3037] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.555419 20268 raft_consensus.cc:504] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.555653 20268 leader_election.cc:302] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.555850 20268 leader_election.cc:288] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.555949 20272 raft_consensus.cc:2781] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.556205 20272 raft_consensus.cc:686] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.556458 20268 ts_tablet_manager.cc:1428] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260811 02:58:21.556551 20272 consensus_queue.cc:237] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.561336 19970 catalog_manager.cc:5495] T 5c447430652a4bf386711fb8e329ca96 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.600682 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59428:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.613914 20165 tablet_service.cc:1467] Processing CreateTablet for tablet e7aec94e498a43cb894252b82cd47433 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.47 [id=40bd6e6729f449ae925026d16c37c620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.614475 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7aec94e498a43cb894252b82cd47433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.619413 20268 tablet_bootstrap.cc:492] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.620971 20268 tablet_bootstrap.cc:654] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.623128 20268 tablet_bootstrap.cc:492] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.623309 20268 ts_tablet_manager.cc:1397] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:21.624110 20268 raft_consensus.cc:348] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.624301 20268 raft_consensus.cc:374] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.624373 20268 raft_consensus.cc:729] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.624549 20268 consensus_queue.cc:260] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.624687 20268 raft_consensus.cc:388] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.624756 20268 raft_consensus.cc:482] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.624835 20268 raft_consensus.cc:3037] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.626761 20268 raft_consensus.cc:504] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.626971 20268 leader_election.cc:302] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.627178 20268 leader_election.cc:288] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.627264 20272 raft_consensus.cc:2781] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.627435 20272 raft_consensus.cc:686] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.627681 20268 ts_tablet_manager.cc:1428] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:21.627672 20272 consensus_queue.cc:237] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.632043 19970 catalog_manager.cc:5495] T e7aec94e498a43cb894252b82cd47433 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.671466 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.678785 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:21.684453 20165 tablet_service.cc:1467] Processing CreateTablet for tablet f01f85da49db482eb6d508163c35b185 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.48 [id=dca6cfc12df84dcca31aee0553f6e511]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.684944 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f01f85da49db482eb6d508163c35b185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.690248 20268 tablet_bootstrap.cc:492] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.691697 20268 tablet_bootstrap.cc:654] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.694114 20268 tablet_bootstrap.cc:492] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.694286 20268 ts_tablet_manager.cc:1397] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:21.694165 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.009s sys 0.004s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":47,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":1314,"drs_written":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1062,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260811 02:58:21.694957 20268 raft_consensus.cc:348] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.695153 20268 raft_consensus.cc:374] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.695183 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.290039
I20260811 02:58:21.695268 20268 raft_consensus.cc:729] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.695518 20268 consensus_queue.cc:260] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.695773 20268 raft_consensus.cc:388] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.695869 20268 raft_consensus.cc:482] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.695986 20268 raft_consensus.cc:3037] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.698808 20268 raft_consensus.cc:504] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.699091 20268 leader_election.cc:302] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.699312 20268 leader_election.cc:288] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.699445 20272 raft_consensus.cc:2781] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.699621 20272 raft_consensus.cc:686] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.699851 20272 consensus_queue.cc:237] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.700189 20268 ts_tablet_manager.cc:1428] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 02:58:21.705227 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260811 02:58:21.708001 19970 catalog_manager.cc:5495] T f01f85da49db482eb6d508163c35b185 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.723721 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.009s sys 0.003s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":1905,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1561,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260811 02:58:21.724987 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.295708
I20260811 02:58:21.740919 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.026s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":14,"cfile_cache_hit_bytes":19193,"cfile_cache_miss":34,"cfile_cache_miss_bytes":29479,"cfile_init":10,"delete_count":0,"delta_iterators_relevant":8,"dirs.queue_time_us":1423,"dirs.run_cpu_time_us":996,"dirs.run_wall_time_us":5542,"drs_written":1,"lbm_read_time_us":11322,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":73,"lbm_write_time_us":1070,"lbm_writes_lt_1ms":31,"mutex_wait_us":445,"num_input_rowsets":2,"peak_mem_usage":13993,"reinsert_count":0,"rows_written":99,"update_count":1}
I20260811 02:58:21.742487 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4449 bytes on disk
I20260811 02:58:21.743338 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.756075 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.017s sys 0.012s Metrics: {"bytes_written":27137,"cfile_cache_hit":13,"cfile_cache_hit_bytes":19276,"cfile_cache_miss":31,"cfile_cache_miss_bytes":29294,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1358,"dirs.run_cpu_time_us":908,"dirs.run_wall_time_us":7127,"drs_written":1,"lbm_read_time_us":1163,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1181,"lbm_writes_lt_1ms":27,"mutex_wait_us":213,"num_input_rowsets":2,"peak_mem_usage":13974,"rows_written":99,"spinlock_wait_cycles":17280}
I20260811 02:58:21.757220 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4466 bytes on disk
I20260811 02:58:21.758002 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":114,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.766821 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:21.769389 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59432:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.783150 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 373a093a44824baaaf6af09432af386a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.49 [id=0d6b5c3daba2403fa37e538d557c4838]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.783677 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 373a093a44824baaaf6af09432af386a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.784415 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.012s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":299,"dirs.run_cpu_time_us":194,"dirs.run_wall_time_us":1388,"drs_written":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4,"lbm_write_time_us":760,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:21.785321 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.269249
I20260811 02:58:21.806363 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.017s sys 0.004s Metrics: {"bytes_written":27137,"cfile_cache_hit":13,"cfile_cache_hit_bytes":19288,"cfile_cache_miss":31,"cfile_cache_miss_bytes":29308,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":744,"dirs.run_cpu_time_us":239,"dirs.run_wall_time_us":1324,"drs_written":1,"lbm_read_time_us":970,"lbm_reads_lt_1ms":67,"lbm_write_time_us":744,"lbm_writes_lt_1ms":27,"mutex_wait_us":95,"num_input_rowsets":2,"peak_mem_usage":14283,"rows_written":99,"spinlock_wait_cycles":768}
I20260811 02:58:21.807333 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4466 bytes on disk
I20260811 02:58:21.807863 20268 tablet_bootstrap.cc:492] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.808065 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.808732 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000007
I20260811 02:58:21.809794 20268 tablet_bootstrap.cc:654] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.812204 20268 tablet_bootstrap.cc:492] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.812414 20268 ts_tablet_manager.cc:1397] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 02:58:21.813122 20268 raft_consensus.cc:348] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.813341 20268 raft_consensus.cc:374] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.813447 20268 raft_consensus.cc:729] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.813758 20268 consensus_queue.cc:260] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.813968 20268 raft_consensus.cc:388] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.814092 20268 raft_consensus.cc:482] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.814213 20268 raft_consensus.cc:3037] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.816375 20268 raft_consensus.cc:504] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.816684 20268 leader_election.cc:302] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.816944 20268 leader_election.cc:288] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.817003 20272 raft_consensus.cc:2781] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.817188 20272 raft_consensus.cc:686] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.817416 20272 consensus_queue.cc:237] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.820003 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":37,"dirs.run_cpu_time_us":251,"dirs.run_wall_time_us":1308,"drs_written":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4,"lbm_write_time_us":713,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:21.820940 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.207937
I20260811 02:58:21.822109 19970 catalog_manager.cc:5495] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.823661 20268 ts_tablet_manager.cc:1428] T 373a093a44824baaaf6af09432af386a P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.011s user 0.004s sys 0.001s
I20260811 02:58:21.848404 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.021s sys 0.003s Metrics: {"bytes_written":334,"cfile_cache_hit":12,"cfile_cache_hit_bytes":19623,"cfile_cache_miss":28,"cfile_cache_miss_bytes":29203,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":905,"dirs.run_cpu_time_us":199,"dirs.run_wall_time_us":1098,"drs_written":1,"lbm_read_time_us":1325,"lbm_reads_lt_1ms":60,"lbm_write_time_us":951,"lbm_writes_lt_1ms":31,"mutex_wait_us":149,"num_input_rowsets":2,"peak_mem_usage":14998,"reinsert_count":0,"rows_written":101,"spinlock_wait_cycles":3840,"update_count":1}
I20260811 02:58:21.849568 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260811 02:58:21.853838 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.004s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":117,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:21.854457 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4534 bytes on disk
I20260811 02:58:21.855288 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4}
I20260811 02:58:21.855962 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260811 02:58:21.861449 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":110,"lbm_read_time_us":98,"lbm_reads_lt_1ms":8,"lbm_write_time_us":131,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260811 02:58:21.886482 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59442:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.899451 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 138dc17cdce34d658f4b38354e8e66fc (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.50 [id=6d2600f63a684a85b2269aef65803cba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.899959 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 138dc17cdce34d658f4b38354e8e66fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.904673 20268 tablet_bootstrap.cc:492] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.906066 20268 tablet_bootstrap.cc:654] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.908075 20268 tablet_bootstrap.cc:492] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.908246 20268 ts_tablet_manager.cc:1397] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:58:21.908981 20268 raft_consensus.cc:348] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.909180 20268 raft_consensus.cc:374] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.909300 20268 raft_consensus.cc:729] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.909672 20268 consensus_queue.cc:260] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.909922 20268 raft_consensus.cc:388] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.910034 20268 raft_consensus.cc:482] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.910146 20268 raft_consensus.cc:3037] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.912177 20268 raft_consensus.cc:504] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.912459 20268 leader_election.cc:302] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.912654 20268 leader_election.cc:288] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.912737 20272 raft_consensus.cc:2781] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.912878 20272 raft_consensus.cc:686] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.913192 20268 ts_tablet_manager.cc:1428] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 02:58:21.913120 20272 consensus_queue.cc:237] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.917954 19968 catalog_manager.cc:5495] T 138dc17cdce34d658f4b38354e8e66fc P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.954689 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59454:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:21.967794 20165 tablet_service.cc:1467] Processing CreateTablet for tablet db6d43c3111d4f9fa703369fdaee2e3e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.51 [id=f251ab626d624b24828366729e66dee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:21.968210 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db6d43c3111d4f9fa703369fdaee2e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:21.973308 20268 tablet_bootstrap.cc:492] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:21.974879 20268 tablet_bootstrap.cc:654] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:21.976727 20268 tablet_bootstrap.cc:492] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:21.976900 20268 ts_tablet_manager.cc:1397] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:58:21.977519 20268 raft_consensus.cc:348] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.977792 20268 raft_consensus.cc:374] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:21.977877 20268 raft_consensus.cc:729] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:21.978144 20268 consensus_queue.cc:260] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.978333 20268 raft_consensus.cc:388] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:21.978438 20268 raft_consensus.cc:482] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:21.978545 20268 raft_consensus.cc:3037] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:21.980453 20268 raft_consensus.cc:504] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.980729 20268 leader_election.cc:302] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:21.980949 20268 leader_election.cc:288] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:21.981003 20272 raft_consensus.cc:2781] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:21.981195 20272 raft_consensus.cc:686] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:21.981437 20272 consensus_queue.cc:237] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:21.981887 20268 ts_tablet_manager.cc:1428] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:21.987957 19970 catalog_manager.cc:5495] T db6d43c3111d4f9fa703369fdaee2e3e P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:21.994376 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260811 02:58:22.006270 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.011s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":220,"dirs.run_wall_time_us":1363,"drs_written":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4,"lbm_write_time_us":691,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:22.007273 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.265574
I20260811 02:58:22.009045 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:22.022737 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":70,"dirs.run_cpu_time_us":250,"dirs.run_wall_time_us":1235,"drs_written":1,"lbm_read_time_us":127,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1064,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:22.023814 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.266381
I20260811 02:58:22.038234 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.024s sys 0.004s Metrics: {"bytes_written":28467,"cfile_cache_hit":13,"cfile_cache_hit_bytes":20393,"cfile_cache_miss":31,"cfile_cache_miss_bytes":30977,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":2090,"dirs.run_cpu_time_us":713,"dirs.run_wall_time_us":8456,"drs_written":1,"lbm_read_time_us":1539,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1109,"lbm_writes_lt_1ms":27,"mutex_wait_us":2129,"num_input_rowsets":2,"peak_mem_usage":15051,"rows_written":105,"spinlock_wait_cycles":83456}
I20260811 02:58:22.039330 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4738 bytes on disk
I20260811 02:58:22.039979 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.049002 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59470:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:22.055495 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.024s sys 0.003s Metrics: {"bytes_written":28467,"cfile_cache_hit":12,"cfile_cache_hit_bytes":20396,"cfile_cache_miss":28,"cfile_cache_miss_bytes":30610,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1088,"dirs.run_cpu_time_us":423,"dirs.run_wall_time_us":2576,"drs_written":1,"lbm_read_time_us":1318,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1311,"lbm_writes_lt_1ms":27,"mutex_wait_us":518,"num_input_rowsets":2,"peak_mem_usage":14998,"rows_written":105}
I20260811 02:58:22.056360 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4738 bytes on disk
I20260811 02:58:22.056985 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.063683 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 0a47c028b6f44c60a1a2fe4f67a588f4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.52 [id=3489d89e00964891b9fbd18cc66cd84a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:22.064177 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a47c028b6f44c60a1a2fe4f67a588f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.069324 20268 tablet_bootstrap.cc:492] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:22.070937 20268 tablet_bootstrap.cc:654] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.072939 20268 tablet_bootstrap.cc:492] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:22.073149 20268 ts_tablet_manager.cc:1397] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260811 02:58:22.074008 20268 raft_consensus.cc:348] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.074267 20268 raft_consensus.cc:374] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.074393 20268 raft_consensus.cc:729] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.074651 20268 consensus_queue.cc:260] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.074860 20268 raft_consensus.cc:388] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.074963 20268 raft_consensus.cc:482] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.075117 20268 raft_consensus.cc:3037] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.077051 20268 raft_consensus.cc:504] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.077337 20268 leader_election.cc:302] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:22.077544 20268 leader_election.cc:288] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.077656 20272 raft_consensus.cc:2781] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.077791 20272 raft_consensus.cc:686] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:22.078047 20272 consensus_queue.cc:237] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.078404 20268 ts_tablet_manager.cc:1428] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:22.082806 19970 catalog_manager.cc:5495] T 0a47c028b6f44c60a1a2fe4f67a588f4 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:22.112687 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:22.126760 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":59,"dirs.run_cpu_time_us":238,"dirs.run_wall_time_us":1473,"drs_written":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1014,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6,"spinlock_wait_cycles":1280}
I20260811 02:58:22.127365 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59486:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:22.127741 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.262804
I20260811 02:58:22.141480 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 846f29baa1e34eaaaa26b74f36c18075 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.53 [id=8dbf283e0fad46a1a5a3ba62120f51fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:22.142081 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 846f29baa1e34eaaaa26b74f36c18075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.147563 20268 tablet_bootstrap.cc:492] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:22.149192 20268 tablet_bootstrap.cc:654] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.151482 20268 tablet_bootstrap.cc:492] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:22.151682 20268 ts_tablet_manager.cc:1397] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:22.152403 20268 raft_consensus.cc:348] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.152623 20268 raft_consensus.cc:374] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.152734 20268 raft_consensus.cc:729] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.152989 20268 consensus_queue.cc:260] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.153174 20268 raft_consensus.cc:388] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.153285 20268 raft_consensus.cc:482] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.153396 20268 raft_consensus.cc:3037] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.155628 20268 raft_consensus.cc:504] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.155898 20268 leader_election.cc:302] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:22.156162 20268 leader_election.cc:288] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.156216 20272 raft_consensus.cc:2781] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.156380 20272 raft_consensus.cc:686] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:22.156661 20268 ts_tablet_manager.cc:1428] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 02:58:22.156615 20272 consensus_queue.cc:237] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.162352 19970 catalog_manager.cc:5495] T 846f29baa1e34eaaaa26b74f36c18075 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:22.163211 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.022s sys 0.003s Metrics: {"bytes_written":28896,"cfile_cache_hit":13,"cfile_cache_hit_bytes":20768,"cfile_cache_miss":31,"cfile_cache_miss_bytes":31564,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":979,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":1681,"drs_written":1,"lbm_read_time_us":1348,"lbm_reads_lt_1ms":67,"lbm_write_time_us":871,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":15511,"rows_written":107,"spinlock_wait_cycles":4992}
I20260811 02:58:22.164366 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260811 02:58:22.182438 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.013s sys 0.000s Metrics: {"bytes_written":335,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":61,"dirs.run_cpu_time_us":162,"dirs.run_wall_time_us":1191,"drs_written":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2537,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:22.183482 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5054 bytes on disk
I20260811 02:58:22.184237 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":176,"lbm_reads_lt_1ms":8}
I20260811 02:58:22.184818 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.463347
I20260811 02:58:22.194053 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":579,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":160,"lbm_read_time_us":289,"lbm_reads_lt_1ms":21,"lbm_write_time_us":326,"lbm_writes_lt_1ms":9,"reinsert_count":0,"spinlock_wait_cycles":13056,"update_count":1}
I20260811 02:58:22.194878 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.216353
I20260811 02:58:22.229212 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.027s sys 0.004s Metrics: {"bytes_written":29338,"cfile_cache_hit":13,"cfile_cache_hit_bytes":21174,"cfile_cache_miss":31,"cfile_cache_miss_bytes":31870,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1027,"dirs.run_cpu_time_us":1067,"dirs.run_wall_time_us":6405,"drs_written":1,"lbm_read_time_us":2116,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1166,"lbm_writes_lt_1ms":27,"mutex_wait_us":321,"num_input_rowsets":2,"peak_mem_usage":16232,"rows_written":109,"thread_start_us":2804,"threads_started":1}
I20260811 02:58:22.230237 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4900 bytes on disk
I20260811 02:58:22.230870 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59490:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:22.231112 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.247064 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 383874ecf403403185a2509f9a46f528 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.54 [id=c0a5320b927441d081428d9de938cd4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:22.247602 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 383874ecf403403185a2509f9a46f528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.252529 20268 tablet_bootstrap.cc:492] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:22.253966 20268 tablet_bootstrap.cc:654] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.256258 20268 tablet_bootstrap.cc:492] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:22.256425 20268 ts_tablet_manager.cc:1397] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:58:22.257122 20268 raft_consensus.cc:348] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.257299 20268 raft_consensus.cc:374] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.257364 20268 raft_consensus.cc:729] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.257537 20268 consensus_queue.cc:260] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.257774 20268 raft_consensus.cc:388] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.257843 20268 raft_consensus.cc:482] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.257933 20268 raft_consensus.cc:3037] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.259830 20268 raft_consensus.cc:504] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.260144 20268 leader_election.cc:302] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:22.260413 20268 leader_election.cc:288] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.260545 20272 raft_consensus.cc:2781] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.260756 20272 raft_consensus.cc:686] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:22.261020 20268 ts_tablet_manager.cc:1428] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:58:22.261073 20272 consensus_queue.cc:237] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.266027 19970 catalog_manager.cc:5495] T 383874ecf403403185a2509f9a46f528 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:22.291115 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:22.307305 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.009s sys 0.004s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":77,"dirs.run_cpu_time_us":244,"dirs.run_wall_time_us":1579,"drs_written":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1232,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:22.308346 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 519 bytes on disk
I20260811 02:58:22.308775 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:22.309149 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.309938 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.235848
I20260811 02:58:22.311566 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59496:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:22.322320 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":116,"dirs.run_cpu_time_us":208,"dirs.run_wall_time_us":1232,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":893,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:22.323226 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.235848
I20260811 02:58:22.331367 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 9ce92a89870f4fb7ae608e20ff1a82a7 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.55 [id=eb1c3cbbdeb041b18e07da30eab33382]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:22.331880 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ce92a89870f4fb7ae608e20ff1a82a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.345832 20268 tablet_bootstrap.cc:492] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:22.347795 20268 tablet_bootstrap.cc:654] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.351593 20268 tablet_bootstrap.cc:492] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:22.351897 20268 ts_tablet_manager.cc:1397] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260811 02:58:22.352829 20268 raft_consensus.cc:348] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.353444 20268 raft_consensus.cc:374] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.353588 20268 raft_consensus.cc:729] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.353999 20268 consensus_queue.cc:260] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.355520 20268 raft_consensus.cc:388] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.355720 20268 raft_consensus.cc:482] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.356016 20268 raft_consensus.cc:3037] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.355464 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.020s sys 0.007s Metrics: {"bytes_written":29768,"cfile_cache_hit":12,"cfile_cache_hit_bytes":21511,"cfile_cache_miss":28,"cfile_cache_miss_bytes":32223,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1528,"dirs.run_cpu_time_us":175,"dirs.run_wall_time_us":1301,"drs_written":1,"lbm_read_time_us":1332,"lbm_reads_lt_1ms":60,"lbm_write_time_us":999,"lbm_writes_lt_1ms":27,"mutex_wait_us":26,"num_input_rowsets":2,"peak_mem_usage":15976,"rows_written":111,"spinlock_wait_cycles":4352}
I20260811 02:58:22.356712 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260811 02:58:22.361539 20268 raft_consensus.cc:504] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.363268 20268 leader_election.cc:302] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:22.363556 20272 raft_consensus.cc:2781] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.363710 20272 raft_consensus.cc:686] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:22.363934 20272 consensus_queue.cc:237] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.375147 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.010s sys 0.005s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":185,"dirs.run_wall_time_us":1019,"drs_written":1,"lbm_read_time_us":153,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1527,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:22.377497 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5213 bytes on disk
I20260811 02:58:22.378849 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":268,"lbm_reads_lt_1ms":8}
I20260811 02:58:22.380921 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.176418
I20260811 02:58:22.381186 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.071s user 0.053s sys 0.004s Metrics: {"bytes_written":29768,"cfile_cache_hit":12,"cfile_cache_hit_bytes":21511,"cfile_cache_miss":28,"cfile_cache_miss_bytes":32223,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":422,"dirs.run_cpu_time_us":775,"dirs.run_wall_time_us":4789,"drs_written":1,"lbm_read_time_us":1287,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1074,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":15976,"rows_written":111,"thread_start_us":29044,"threads_started":1}
I20260811 02:58:22.382366 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000021
I20260811 02:58:22.396492 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.007s sys 0.004s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":63,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1301,"drs_written":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1021,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:22.397514 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5213 bytes on disk
I20260811 02:58:22.398525 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":175,"lbm_reads_lt_1ms":8}
I20260811 02:58:22.399330 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.176418
I20260811 02:58:22.402614 20268 leader_election.cc:288] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.404429 20268 ts_tablet_manager.cc:1428] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.052s user 0.004s sys 0.003s
I20260811 02:58:22.408517 19968 catalog_manager.cc:5495] T 9ce92a89870f4fb7ae608e20ff1a82a7 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:22.422161 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.030s sys 0.000s Metrics: {"bytes_written":30175,"cfile_cache_hit":12,"cfile_cache_hit_bytes":21843,"cfile_cache_miss":28,"cfile_cache_miss_bytes":32469,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":5644,"dirs.run_cpu_time_us":1059,"dirs.run_wall_time_us":4612,"drs_written":1,"lbm_read_time_us":1268,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1231,"lbm_writes_lt_1ms":27,"mutex_wait_us":8644,"num_input_rowsets":2,"peak_mem_usage":16744,"rows_written":113,"spinlock_wait_cycles":17664}
I20260811 02:58:22.423117 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5039 bytes on disk
I20260811 02:58:22.423804 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.424472 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260811 02:58:22.430212 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.027s sys 0.000s Metrics: {"bytes_written":30175,"cfile_cache_hit":12,"cfile_cache_hit_bytes":21843,"cfile_cache_miss":28,"cfile_cache_miss_bytes":32469,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":947,"dirs.run_cpu_time_us":1007,"dirs.run_wall_time_us":5394,"drs_written":1,"lbm_read_time_us":1494,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1128,"lbm_writes_lt_1ms":27,"mutex_wait_us":155,"num_input_rowsets":2,"peak_mem_usage":16744,"rows_written":113,"spinlock_wait_cycles":2432,"thread_start_us":678,"threads_started":1}
I20260811 02:58:22.430608 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.004s sys 0.000s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":147,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:22.431286 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5039 bytes on disk
I20260811 02:58:22.431900 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.432554 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260811 02:58:22.438007 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.005s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":195,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:22.438699 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260811 02:58:22.442414 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.003s user 0.001s sys 0.002s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":109,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:22.442904 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260811 02:58:22.449476 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.004s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":73,"lbm_read_time_us":126,"lbm_reads_lt_1ms":12,"lbm_write_time_us":121,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260811 02:58:22.476855 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59506:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:22.482506 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260811 02:58:22.491683 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 4109cc4ab0c644299ca16502de07c8d6 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.56 [id=10a86d7e84894bdc93eeb86d0c273c5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:22.492205 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4109cc4ab0c644299ca16502de07c8d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.498790 20268 tablet_bootstrap.cc:492] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:22.500222 20268 tablet_bootstrap.cc:654] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.500074 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.008s sys 0.003s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":1257,"dirs.run_cpu_time_us":259,"dirs.run_wall_time_us":1381,"drs_written":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1242,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:22.501005 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.245353
I20260811 02:58:22.502377 20268 tablet_bootstrap.cc:492] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:22.502544 20268 ts_tablet_manager.cc:1397] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:22.503182 20268 raft_consensus.cc:348] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.503373 20268 raft_consensus.cc:374] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.503494 20268 raft_consensus.cc:729] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.503747 20268 consensus_queue.cc:260] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.504051 20268 raft_consensus.cc:388] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.504150 20268 raft_consensus.cc:482] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.504279 20268 raft_consensus.cc:3037] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.506321 20268 raft_consensus.cc:504] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.506631 20268 leader_election.cc:302] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:22.506843 20268 leader_election.cc:288] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.506961 20272 raft_consensus.cc:2781] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.507136 20272 raft_consensus.cc:686] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:22.507506 20268 ts_tablet_manager.cc:1428] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260811 02:58:22.507443 20272 consensus_queue.cc:237] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.510291 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1062,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":52,"lbm_read_time_us":338,"lbm_reads_lt_1ms":25,"lbm_write_time_us":338,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:22.511050 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.241954
I20260811 02:58:22.513839 19968 catalog_manager.cc:5495] T 4109cc4ab0c644299ca16502de07c8d6 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:22.547953 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.027s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":22247,"cfile_cache_miss":31,"cfile_cache_miss_bytes":33381,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":894,"dirs.run_cpu_time_us":252,"dirs.run_wall_time_us":1394,"drs_written":1,"lbm_read_time_us":1436,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1028,"lbm_writes_lt_1ms":31,"mutex_wait_us":144,"num_input_rowsets":2,"peak_mem_usage":16488,"reinsert_count":0,"rows_written":115,"spinlock_wait_cycles":2304,"update_count":1}
I20260811 02:58:22.549067 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5142 bytes on disk
I20260811 02:58:22.549791 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.557026 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59508:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:22.575085 20165 tablet_service.cc:1467] Processing CreateTablet for tablet e8ebf34d70a24f6093a7ab9cf137753d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.57 [id=1415a110cb624f94a3f669ed573ac1a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:22.575637 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8ebf34d70a24f6093a7ab9cf137753d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.580478 20268 tablet_bootstrap.cc:492] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:22.582022 20268 tablet_bootstrap.cc:654] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.584169 20268 tablet_bootstrap.cc:492] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:22.584396 20268 ts_tablet_manager.cc:1397] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:22.585124 20268 raft_consensus.cc:348] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.585340 20268 raft_consensus.cc:374] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.585425 20268 raft_consensus.cc:729] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.585714 20268 consensus_queue.cc:260] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.585939 20268 raft_consensus.cc:388] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.586051 20268 raft_consensus.cc:482] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.586149 20268 raft_consensus.cc:3037] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.588472 20268 raft_consensus.cc:504] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.588682 20268 leader_election.cc:302] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:22.588923 20268 leader_election.cc:288] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.588986 20272 raft_consensus.cc:2781] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.589162 20272 raft_consensus.cc:686] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:22.589485 20268 ts_tablet_manager.cc:1428] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 02:58:22.589417 20272 consensus_queue.cc:237] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.597949 19968 catalog_manager.cc:5495] T e8ebf34d70a24f6093a7ab9cf137753d P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:22.639146 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59516:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:22.652519 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 86a5348b0e61421caf3c8497c5df6256 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.58 [id=d2236257976f49ac936d49d7fa389a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:22.653077 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a5348b0e61421caf3c8497c5df6256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.658344 20268 tablet_bootstrap.cc:492] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:22.660045 20268 tablet_bootstrap.cc:654] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.662463 20268 tablet_bootstrap.cc:492] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:22.662649 20268 ts_tablet_manager.cc:1397] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:22.663296 20268 raft_consensus.cc:348] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.663462 20268 raft_consensus.cc:374] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.663529 20268 raft_consensus.cc:729] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.663784 20268 consensus_queue.cc:260] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.664024 20268 raft_consensus.cc:388] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.664129 20268 raft_consensus.cc:482] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.664259 20268 raft_consensus.cc:3037] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.666296 20268 raft_consensus.cc:504] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.666570 20268 leader_election.cc:302] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:22.666792 20268 leader_election.cc:288] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.666961 20272 raft_consensus.cc:2781] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.667147 20272 raft_consensus.cc:686] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:22.667333 20268 ts_tablet_manager.cc:1428] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:22.667414 20272 consensus_queue.cc:237] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.672693 19968 catalog_manager.cc:5495] T 86a5348b0e61421caf3c8497c5df6256 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:22.681560 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260811 02:58:22.694548 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.003s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":144,"dirs.run_wall_time_us":1255,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1003,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:22.695546 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 520 bytes on disk
I20260811 02:58:22.696225 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.696877 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.259438
I20260811 02:58:22.700604 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260811 02:58:22.716626 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.007s sys 0.007s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":573,"dirs.run_cpu_time_us":185,"dirs.run_wall_time_us":1234,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1029,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:22.718214 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 520 bytes on disk
I20260811 02:58:22.719033 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.719882 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.259320
I20260811 02:58:22.730297 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.026s sys 0.004s Metrics: {"bytes_written":31514,"cfile_cache_hit":13,"cfile_cache_hit_bytes":22988,"cfile_cache_miss":31,"cfile_cache_miss_bytes":34702,"cfile_init":7,"delta_iterators_relevant":6,"dirs.queue_time_us":1349,"dirs.run_cpu_time_us":925,"dirs.run_wall_time_us":4912,"drs_written":1,"lbm_read_time_us":1400,"lbm_reads_lt_1ms":59,"lbm_write_time_us":1002,"lbm_writes_lt_1ms":27,"mutex_wait_us":21,"num_input_rowsets":2,"peak_mem_usage":17820,"rows_written":119}
I20260811 02:58:22.731393 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5325 bytes on disk
I20260811 02:58:22.732131 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.734813 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59520:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:22.750576 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 79da34f542f649f6864ceaaddfeb7f0e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.59 [id=8b1357a0b2814d4c9a114158d1ef1693]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:22.751124 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79da34f542f649f6864ceaaddfeb7f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.757725 20268 tablet_bootstrap.cc:492] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:22.759446 20268 tablet_bootstrap.cc:654] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.762135 20268 tablet_bootstrap.cc:492] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:22.762347 20268 ts_tablet_manager.cc:1397] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:22.763165 20268 raft_consensus.cc:348] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.763439 20268 raft_consensus.cc:374] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.763540 20268 raft_consensus.cc:729] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.763808 20268 consensus_queue.cc:260] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.764039 20268 raft_consensus.cc:388] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.764178 20268 raft_consensus.cc:482] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.764323 20268 raft_consensus.cc:3037] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.766330 20268 raft_consensus.cc:504] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.766597 20268 leader_election.cc:302] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:22.766853 20268 leader_election.cc:288] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.766897 20272 raft_consensus.cc:2781] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.767123 20272 raft_consensus.cc:686] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:22.767350 20272 consensus_queue.cc:237] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.767680 20268 ts_tablet_manager.cc:1428] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:22.768220 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.026s sys 0.008s Metrics: {"bytes_written":31514,"cfile_cache_hit":13,"cfile_cache_hit_bytes":22988,"cfile_cache_miss":31,"cfile_cache_miss_bytes":34814,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1269,"dirs.run_cpu_time_us":1153,"dirs.run_wall_time_us":8633,"drs_written":1,"lbm_read_time_us":1346,"lbm_reads_lt_1ms":63,"lbm_write_time_us":2549,"lbm_writes_lt_1ms":27,"mutex_wait_us":55,"num_input_rowsets":2,"peak_mem_usage":18027,"rows_written":119,"thread_start_us":5056,"threads_started":1}
I20260811 02:58:22.769173 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000015
I20260811 02:58:22.777927 19968 catalog_manager.cc:5495] T 79da34f542f649f6864ceaaddfeb7f0e P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:22.781155 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":72,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":4169,"drs_written":1,"lbm_read_time_us":69,"lbm_reads_lt_1ms":4,"lbm_write_time_us":815,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:22.782594 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.200872
I20260811 02:58:22.800850 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260811 02:58:22.814415 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":70,"dirs.run_cpu_time_us":255,"dirs.run_wall_time_us":1251,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1034,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:22.814534 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.025s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":23323,"cfile_cache_miss":28,"cfile_cache_miss_bytes":34641,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":582,"dirs.run_cpu_time_us":522,"dirs.run_wall_time_us":2789,"drs_written":1,"lbm_read_time_us":1201,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1281,"lbm_writes_lt_1ms":31,"mutex_wait_us":141,"num_input_rowsets":2,"peak_mem_usage":17973,"reinsert_count":0,"rows_written":121,"update_count":1}
I20260811 02:58:22.815457 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 520 bytes on disk
I20260811 02:58:22.816215 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.816370 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5398 bytes on disk
I20260811 02:58:22.816953 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.258674
I20260811 02:58:22.816957 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.830442 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59528:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:22.849953 20165 tablet_service.cc:1467] Processing CreateTablet for tablet a5c063eba685437189e9a4fbde647552 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.60 [id=22cda2285fcb4a60a88c4cea8fe216ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:22.850474 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c063eba685437189e9a4fbde647552. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.850850 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.028s sys 0.002s Metrics: {"bytes_written":31961,"cfile_cache_hit":13,"cfile_cache_hit_bytes":23353,"cfile_cache_miss":31,"cfile_cache_miss_bytes":35271,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1182,"dirs.run_cpu_time_us":725,"dirs.run_wall_time_us":3920,"drs_written":1,"lbm_read_time_us":1388,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1199,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":18289,"rows_written":121}
I20260811 02:58:22.852063 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260811 02:58:22.855973 20268 tablet_bootstrap.cc:492] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:22.857703 20268 tablet_bootstrap.cc:654] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.860282 20268 tablet_bootstrap.cc:492] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:22.860455 20268 ts_tablet_manager.cc:1397] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:22.861292 20268 raft_consensus.cc:348] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.861541 20268 raft_consensus.cc:374] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.861682 20268 raft_consensus.cc:729] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.862003 20268 consensus_queue.cc:260] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.862237 20268 raft_consensus.cc:388] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.862339 20268 raft_consensus.cc:482] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.862443 20268 raft_consensus.cc:3037] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.863919 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.005s sys 0.004s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":251,"dirs.run_wall_time_us":1296,"drs_written":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4,"lbm_write_time_us":746,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:22.864972 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.229925
I20260811 02:58:22.865118 20268 raft_consensus.cc:504] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.865453 20268 leader_election.cc:302] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:22.865734 20268 leader_election.cc:288] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.865837 20272 raft_consensus.cc:2781] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.866050 20272 raft_consensus.cc:686] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:22.866288 20268 ts_tablet_manager.cc:1428] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 02:58:22.866312 20272 consensus_queue.cc:237] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.873234 19968 catalog_manager.cc:5495] T a5c063eba685437189e9a4fbde647552 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:22.897661 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.029s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":23698,"cfile_cache_miss":28,"cfile_cache_miss_bytes":35194,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":993,"dirs.run_cpu_time_us":186,"dirs.run_wall_time_us":1646,"drs_written":1,"lbm_read_time_us":1214,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1114,"lbm_writes_lt_1ms":31,"mutex_wait_us":287,"num_input_rowsets":2,"peak_mem_usage":18229,"reinsert_count":0,"rows_written":123,"update_count":1}
I20260811 02:58:22.898801 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5476 bytes on disk
I20260811 02:58:22.899537 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4}
I20260811 02:58:22.922359 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59542:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.61"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:22.936728 20165 tablet_service.cc:1467] Processing CreateTablet for tablet f5e1351951cf4ca08428fe6f73181fe7 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.61 [id=78fb33a5788241859f0e5979d9bfe4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:22.937229 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5e1351951cf4ca08428fe6f73181fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.942008 20268 tablet_bootstrap.cc:492] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:22.943536 20268 tablet_bootstrap.cc:654] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.945977 20268 tablet_bootstrap.cc:492] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:22.946120 20268 ts_tablet_manager.cc:1397] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:22.946694 20268 raft_consensus.cc:348] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.946893 20268 raft_consensus.cc:374] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.946967 20268 raft_consensus.cc:729] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.947155 20268 consensus_queue.cc:260] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.947291 20268 raft_consensus.cc:388] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.947357 20268 raft_consensus.cc:482] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.947423 20268 raft_consensus.cc:3037] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.949719 20268 raft_consensus.cc:504] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.949939 20268 leader_election.cc:302] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:22.950122 20268 leader_election.cc:288] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.950239 20272 raft_consensus.cc:2781] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.950466 20272 raft_consensus.cc:686] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:22.950600 20268 ts_tablet_manager.cc:1428] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:58:22.950690 20272 consensus_queue.cc:237] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:22.955819 19968 catalog_manager.cc:5495] T f5e1351951cf4ca08428fe6f73181fe7 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:22.983238 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:22.997426 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.009s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":87,"dirs.run_cpu_time_us":195,"dirs.run_wall_time_us":1308,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":837,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6,"spinlock_wait_cycles":14208}
I20260811 02:58:22.998468 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 523 bytes on disk
I20260811 02:58:22.999239 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":135,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.000815 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.257783
I20260811 02:58:23.007268 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59546:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:23.021944 20165 tablet_service.cc:1467] Processing CreateTablet for tablet bb2d51829f4c41558d3ab99a4fc6a6f4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.62 [id=f6482310f1b94340a85c1f348bdf2dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:23.022863 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb2d51829f4c41558d3ab99a4fc6a6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.028162 20268 tablet_bootstrap.cc:492] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:23.029654 20268 tablet_bootstrap.cc:654] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.031368 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.024s sys 0.003s Metrics: {"bytes_written":32832,"cfile_cache_hit":12,"cfile_cache_hit_bytes":24096,"cfile_cache_miss":28,"cfile_cache_miss_bytes":35986,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1052,"dirs.run_cpu_time_us":1131,"dirs.run_wall_time_us":7040,"drs_written":1,"lbm_read_time_us":1158,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1144,"lbm_writes_lt_1ms":27,"mutex_wait_us":351,"num_input_rowsets":2,"peak_mem_usage":18748,"rows_written":125,"spinlock_wait_cycles":4736,"thread_start_us":296,"threads_started":1}
I20260811 02:58:23.032291 20268 tablet_bootstrap.cc:492] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:23.032334 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5577 bytes on disk
I20260811 02:58:23.032457 20268 ts_tablet_manager.cc:1397] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:23.032948 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.033141 20268 raft_consensus.cc:348] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.033366 20268 raft_consensus.cc:374] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.033497 20268 raft_consensus.cc:729] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.033546 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260811 02:58:23.033776 20268 consensus_queue.cc:260] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.034037 20268 raft_consensus.cc:388] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.034147 20268 raft_consensus.cc:482] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.034267 20268 raft_consensus.cc:3037] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.036476 20268 raft_consensus.cc:504] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.036777 20268 leader_election.cc:302] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:23.037014 20268 leader_election.cc:288] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.037101 20272 raft_consensus.cc:2781] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.037528 20268 ts_tablet_manager.cc:1428] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 02:58:23.037912 20272 raft_consensus.cc:686] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:23.038208 20272 consensus_queue.cc:237] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.044425 19970 catalog_manager.cc:5495] T bb2d51829f4c41558d3ab99a4fc6a6f4 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:23.045382 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.007s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":42,"dirs.run_cpu_time_us":170,"dirs.run_wall_time_us":4318,"drs_written":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4,"lbm_write_time_us":807,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:23.046221 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:23.047845 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.048560 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.198109
I20260811 02:58:23.067843 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:23.078471 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.008s sys 0.002s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":71,"dirs.run_cpu_time_us":229,"dirs.run_wall_time_us":1266,"drs_written":1,"lbm_read_time_us":52,"lbm_reads_lt_1ms":4,"lbm_write_time_us":794,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:23.079272 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 518 bytes on disk
I20260811 02:58:23.079851 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.080353 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.021s sys 0.007s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":24438,"cfile_cache_miss":28,"cfile_cache_miss_bytes":36268,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":426,"dirs.run_cpu_time_us":226,"dirs.run_wall_time_us":1310,"drs_written":1,"lbm_read_time_us":1108,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1274,"lbm_writes_lt_1ms":32,"mutex_wait_us":145,"num_input_rowsets":2,"peak_mem_usage":18835,"reinsert_count":0,"rows_written":127,"update_count":1}
I20260811 02:58:23.080652 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.256479
I20260811 02:58:23.081642 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5645 bytes on disk
I20260811 02:58:23.082333 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.113662 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.024s sys 0.008s Metrics: {"bytes_written":33287,"cfile_cache_hit":13,"cfile_cache_hit_bytes":24468,"cfile_cache_miss":31,"cfile_cache_miss_bytes":36908,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":923,"dirs.run_cpu_time_us":254,"dirs.run_wall_time_us":1382,"drs_written":1,"lbm_read_time_us":1744,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1128,"lbm_writes_lt_1ms":28,"mutex_wait_us":272,"num_input_rowsets":2,"peak_mem_usage":19057,"rows_written":127,"spinlock_wait_cycles":3584}
I20260811 02:58:23.114683 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5659 bytes on disk
I20260811 02:58:23.115242 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.117390 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59550:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:23.130963 20165 tablet_service.cc:1467] Processing CreateTablet for tablet fa8d6c24c594448c89588f17e28d2cc1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.63 [id=dd8d465b4f98464badd045101083c2f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:23.131443 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa8d6c24c594448c89588f17e28d2cc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.136358 20268 tablet_bootstrap.cc:492] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:23.137800 20268 tablet_bootstrap.cc:654] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.140329 20268 tablet_bootstrap.cc:492] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:23.140538 20268 ts_tablet_manager.cc:1397] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260811 02:58:23.141245 20268 raft_consensus.cc:348] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.141474 20268 raft_consensus.cc:374] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.141568 20268 raft_consensus.cc:729] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.141901 20268 consensus_queue.cc:260] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.142153 20268 raft_consensus.cc:388] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.142297 20268 raft_consensus.cc:482] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.142426 20268 raft_consensus.cc:3037] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.144863 20268 raft_consensus.cc:504] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.145085 20268 leader_election.cc:302] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:23.145314 20268 leader_election.cc:288] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.145383 20272 raft_consensus.cc:2781] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.145598 20272 raft_consensus.cc:686] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:23.145969 20268 ts_tablet_manager.cc:1428] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:58:23.145879 20272 consensus_queue.cc:237] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.150583 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:23.152141 19968 catalog_manager.cc:5495] T fa8d6c24c594448c89588f17e28d2cc1 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:23.167644 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.010s sys 0.002s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":202,"dirs.run_wall_time_us":1285,"drs_written":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1202,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:23.168632 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.261696
I20260811 02:58:23.198928 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.015s sys 0.012s Metrics: {"bytes_written":33730,"cfile_cache_hit":14,"cfile_cache_hit_bytes":24830,"cfile_cache_miss":34,"cfile_cache_miss_bytes":37804,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":1141,"dirs.run_cpu_time_us":1438,"dirs.run_wall_time_us":6340,"drs_written":1,"lbm_read_time_us":1432,"lbm_reads_lt_1ms":74,"lbm_write_time_us":901,"lbm_writes_lt_1ms":28,"mutex_wait_us":264,"num_input_rowsets":2,"peak_mem_usage":19366,"rows_written":129,"spinlock_wait_cycles":193920,"thread_start_us":768,"threads_started":1}
I20260811 02:58:23.199807 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5738 bytes on disk
I20260811 02:58:23.200505 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.203225 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59564:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:23.218605 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 8442e81a975a4dbbb78b32286911f155 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.64 [id=1ef85de5dba24648b5b6a030d5a2f085]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:23.219018 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8442e81a975a4dbbb78b32286911f155. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.223500 20268 tablet_bootstrap.cc:492] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:23.225114 20268 tablet_bootstrap.cc:654] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.228092 20268 tablet_bootstrap.cc:492] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:23.228251 20268 ts_tablet_manager.cc:1397] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 02:58:23.229065 20268 raft_consensus.cc:348] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.229282 20268 raft_consensus.cc:374] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.229374 20268 raft_consensus.cc:729] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.229592 20268 consensus_queue.cc:260] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.229907 20268 raft_consensus.cc:388] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.229991 20268 raft_consensus.cc:482] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.230123 20268 raft_consensus.cc:3037] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.233180 20268 raft_consensus.cc:504] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.233486 20268 leader_election.cc:302] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:23.233726 20268 leader_election.cc:288] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.233844 20272 raft_consensus.cc:2781] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.234021 20272 raft_consensus.cc:686] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:23.234277 20272 consensus_queue.cc:237] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.235057 20268 ts_tablet_manager.cc:1428] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 02:58:23.240257 19968 catalog_manager.cc:5495] T 8442e81a975a4dbbb78b32286911f155 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:23.284260 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59574:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:23.298082 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 2e0af8606df443b69610e4060325dd35 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.65 [id=5543fcfcdc224e81919719458398754d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:23.298717 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e0af8606df443b69610e4060325dd35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.303436 20268 tablet_bootstrap.cc:492] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:23.305084 20268 tablet_bootstrap.cc:654] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.307647 20268 tablet_bootstrap.cc:492] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:23.307835 20268 ts_tablet_manager.cc:1397] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:23.308665 20268 raft_consensus.cc:348] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.308895 20268 raft_consensus.cc:374] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.308972 20268 raft_consensus.cc:729] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.309247 20268 consensus_queue.cc:260] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.309476 20268 raft_consensus.cc:388] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.309585 20268 raft_consensus.cc:482] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.309715 20268 raft_consensus.cc:3037] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.312887 20268 raft_consensus.cc:504] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.313207 20268 leader_election.cc:302] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:23.313514 20272 raft_consensus.cc:2781] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.313701 20272 raft_consensus.cc:686] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:23.313949 20272 consensus_queue.cc:237] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.315032 20268 leader_election.cc:288] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.315659 20268 ts_tablet_manager.cc:1428] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.008s user 0.001s sys 0.005s
I20260811 02:58:23.322271 19968 catalog_manager.cc:5495] T 2e0af8606df443b69610e4060325dd35 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:23.333380 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:23.350198 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.014s sys 0.000s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":65,"dirs.run_cpu_time_us":187,"dirs.run_wall_time_us":1505,"drs_written":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1495,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:23.352509 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.264532
I20260811 02:58:23.366230 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:23.381795 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.012s sys 0.002s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":83,"dirs.run_cpu_time_us":149,"dirs.run_wall_time_us":1102,"drs_written":1,"lbm_read_time_us":162,"lbm_reads_lt_1ms":4,"lbm_write_time_us":748,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:23.383265 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.255602
I20260811 02:58:23.391368 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.023s sys 0.014s Metrics: {"bytes_written":34603,"cfile_cache_hit":14,"cfile_cache_hit_bytes":25604,"cfile_cache_miss":34,"cfile_cache_miss_bytes":38942,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":689,"dirs.run_cpu_time_us":1039,"dirs.run_wall_time_us":6445,"drs_written":1,"lbm_read_time_us":1516,"lbm_reads_lt_1ms":74,"lbm_write_time_us":1042,"lbm_writes_lt_1ms":28,"mutex_wait_us":55,"num_input_rowsets":2,"peak_mem_usage":20049,"rows_written":133,"spinlock_wait_cycles":7040,"thread_start_us":7016,"threads_started":1}
I20260811 02:58:23.392306 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5916 bytes on disk
I20260811 02:58:23.392982 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.412735 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59582:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:23.425372 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.030s sys 0.004s Metrics: {"bytes_written":34603,"cfile_cache_hit":12,"cfile_cache_hit_bytes":25576,"cfile_cache_miss":28,"cfile_cache_miss_bytes":38188,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":492,"dirs.run_cpu_time_us":170,"dirs.run_wall_time_us":1326,"drs_written":1,"lbm_read_time_us":1859,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1841,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":19982,"rows_written":133}
I20260811 02:58:23.426468 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260811 02:58:23.429723 20165 tablet_service.cc:1467] Processing CreateTablet for tablet f636105bd2024007b583f934d6da7b53 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.66 [id=5b6cf0e630394893aa7f37a445a99c93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:23.430218 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f636105bd2024007b583f934d6da7b53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.436661 20268 tablet_bootstrap.cc:492] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:23.438551 20268 tablet_bootstrap.cc:654] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.439690 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.003s sys 0.008s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":81,"dirs.run_cpu_time_us":230,"dirs.run_wall_time_us":1329,"drs_written":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1098,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"spinlock_wait_cycles":15744,"update_count":1}
I20260811 02:58:23.440488 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260811 02:58:23.440829 20268 tablet_bootstrap.cc:492] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:23.441017 20268 ts_tablet_manager.cc:1397] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:23.441972 20268 raft_consensus.cc:348] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.442194 20268 raft_consensus.cc:374] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.442354 20268 raft_consensus.cc:729] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.442651 20268 consensus_queue.cc:260] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.442893 20268 raft_consensus.cc:388] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.443014 20268 raft_consensus.cc:482] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.443157 20268 raft_consensus.cc:3037] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.445307 20268 raft_consensus.cc:504] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.445727 20268 leader_election.cc:302] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:23.445973 20268 leader_election.cc:288] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.446085 20272 raft_consensus.cc:2781] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.446242 20272 raft_consensus.cc:686] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:23.446497 20272 consensus_queue.cc:237] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.447305 20268 ts_tablet_manager.cc:1428] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:58:23.449755 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.006s sys 0.003s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":108,"lbm_read_time_us":475,"lbm_reads_lt_1ms":25,"lbm_write_time_us":245,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:23.450438 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.235372
I20260811 02:58:23.452400 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:23.452939 19970 catalog_manager.cc:5495] T f636105bd2024007b583f934d6da7b53 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:23.472378 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.007s sys 0.007s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":80,"dirs.run_cpu_time_us":253,"dirs.run_wall_time_us":1290,"drs_written":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1307,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:23.473436 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 527 bytes on disk
I20260811 02:58:23.474165 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.474862 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.263335
I20260811 02:58:23.493224 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.034s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":25951,"cfile_cache_miss":31,"cfile_cache_miss_bytes":38481,"cfile_init":10,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":2451,"dirs.run_cpu_time_us":1108,"dirs.run_wall_time_us":7959,"drs_written":1,"lbm_read_time_us":2192,"lbm_reads_lt_1ms":71,"lbm_write_time_us":1424,"lbm_writes_lt_1ms":32,"mutex_wait_us":200,"num_input_rowsets":2,"peak_mem_usage":20081,"reinsert_count":0,"rows_written":135,"spinlock_wait_cycles":20736,"thread_start_us":4283,"threads_started":1,"update_count":1}
I20260811 02:58:23.494431 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5987 bytes on disk
I20260811 02:58:23.495289 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.504570 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.022s sys 0.004s Metrics: {"bytes_written":35031,"cfile_cache_hit":13,"cfile_cache_hit_bytes":25948,"cfile_cache_miss":31,"cfile_cache_miss_bytes":39072,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1106,"dirs.run_cpu_time_us":159,"dirs.run_wall_time_us":2198,"drs_written":1,"lbm_read_time_us":3095,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":62,"lbm_write_time_us":826,"lbm_writes_lt_1ms":28,"mutex_wait_us":248,"num_input_rowsets":2,"peak_mem_usage":20286,"rows_written":135,"spinlock_wait_cycles":9600}
I20260811 02:58:23.505532 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5998 bytes on disk
I20260811 02:58:23.506347 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.518775 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59586:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:23.532255 20165 tablet_service.cc:1467] Processing CreateTablet for tablet fffccec7b3e3472f845e52ef0a2b6f13 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.67 [id=fcb242fa6ad64e3ea2e51bd5b7d4c1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:23.532799 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fffccec7b3e3472f845e52ef0a2b6f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.537443 20268 tablet_bootstrap.cc:492] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:23.539014 20268 tablet_bootstrap.cc:654] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.540997 20268 tablet_bootstrap.cc:492] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:23.541193 20268 ts_tablet_manager.cc:1397] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:23.541944 20268 raft_consensus.cc:348] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.542163 20268 raft_consensus.cc:374] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.542263 20268 raft_consensus.cc:729] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.542518 20268 consensus_queue.cc:260] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.542773 20268 raft_consensus.cc:388] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.542888 20268 raft_consensus.cc:482] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.542994 20268 raft_consensus.cc:3037] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.545094 20268 raft_consensus.cc:504] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.545313 20268 leader_election.cc:302] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:23.545526 20268 leader_election.cc:288] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.545585 20272 raft_consensus.cc:2781] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.545814 20272 raft_consensus.cc:686] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:23.546075 20268 ts_tablet_manager.cc:1428] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:23.546108 20272 consensus_queue.cc:237] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.551438 19970 catalog_manager.cc:5495] T fffccec7b3e3472f845e52ef0a2b6f13 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:23.595703 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59598:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:23.611121 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 1fdd6279af9f45b5b09989b304a3e7d5 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.68 [id=c0508b94a9fe465084d6d41e6a89eba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:23.611655 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fdd6279af9f45b5b09989b304a3e7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.616554 20268 tablet_bootstrap.cc:492] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:23.618160 20268 tablet_bootstrap.cc:654] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.620357 20268 tablet_bootstrap.cc:492] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:23.620492 20268 ts_tablet_manager.cc:1397] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:23.621135 20268 raft_consensus.cc:348] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.621389 20268 raft_consensus.cc:374] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.621488 20268 raft_consensus.cc:729] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.621764 20268 consensus_queue.cc:260] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.622041 20268 raft_consensus.cc:388] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.622140 20268 raft_consensus.cc:482] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.622262 20268 raft_consensus.cc:3037] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.624377 20268 raft_consensus.cc:504] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.624650 20268 leader_election.cc:302] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:23.624841 20268 leader_election.cc:288] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.624933 20272 raft_consensus.cc:2781] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.625072 20272 raft_consensus.cc:686] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:23.625394 20268 ts_tablet_manager.cc:1428] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 02:58:23.625322 20272 consensus_queue.cc:237] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.631278 19970 catalog_manager.cc:5495] T 1fdd6279af9f45b5b09989b304a3e7d5 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:23.643998 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260811 02:58:23.655725 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":73,"dirs.run_cpu_time_us":240,"dirs.run_wall_time_us":1427,"drs_written":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4,"lbm_write_time_us":994,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:23.656611 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 522 bytes on disk
I20260811 02:58:23.657235 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.657958 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.251369
I20260811 02:58:23.678966 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59606:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:23.689531 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.024s sys 0.004s Metrics: {"bytes_written":35903,"cfile_cache_hit":12,"cfile_cache_hit_bytes":26691,"cfile_cache_miss":28,"cfile_cache_miss_bytes":39755,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":806,"dirs.run_cpu_time_us":231,"dirs.run_wall_time_us":1365,"drs_written":1,"lbm_read_time_us":1060,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1117,"lbm_writes_lt_1ms":28,"mutex_wait_us":19,"num_input_rowsets":2,"peak_mem_usage":20745,"rows_written":139,"spinlock_wait_cycles":4480}
I20260811 02:58:23.690835 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6173 bytes on disk
I20260811 02:58:23.691787 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.692539 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260811 02:58:23.695218 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 4109b4c60bf042e2b20630263a10ff44 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.69 [id=19a31c02dbae4e2293bd9a6a569d0f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:23.695750 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4109b4c60bf042e2b20630263a10ff44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.700887 20268 tablet_bootstrap.cc:492] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:23.702787 20268 tablet_bootstrap.cc:654] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.705571 20268 tablet_bootstrap.cc:492] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:23.705797 20268 ts_tablet_manager.cc:1397] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260811 02:58:23.705662 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.000s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":34,"dirs.run_cpu_time_us":190,"dirs.run_wall_time_us":1304,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":931,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"spinlock_wait_cycles":7296,"update_count":1}
I20260811 02:58:23.706569 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 167 bytes on disk
I20260811 02:58:23.706526 20268 raft_consensus.cc:348] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.706769 20268 raft_consensus.cc:374] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.706852 20268 raft_consensus.cc:729] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.707108 20268 consensus_queue.cc:260] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.707222 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.707301 20268 raft_consensus.cc:388] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.707403 20268 raft_consensus.cc:482] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.707531 20268 raft_consensus.cc:3037] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.707971 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260811 02:58:23.709759 20268 raft_consensus.cc:504] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.710019 20268 leader_election.cc:302] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:23.710273 20268 leader_election.cc:288] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.710356 20272 raft_consensus.cc:2781] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.710556 20272 raft_consensus.cc:686] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:23.710786 20268 ts_tablet_manager.cc:1428] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:23.710824 20272 consensus_queue.cc:237] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.715921 19970 catalog_manager.cc:5495] T 4109b4c60bf042e2b20630263a10ff44 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:23.721364 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"lbm_read_time_us":391,"lbm_reads_lt_1ms":21,"lbm_write_time_us":391,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:23.722065 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.206325
I20260811 02:58:23.746344 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260811 02:58:23.755231 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.025s sys 0.007s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":27056,"cfile_cache_miss":31,"cfile_cache_miss_bytes":40106,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1118,"dirs.run_cpu_time_us":217,"dirs.run_wall_time_us":1362,"drs_written":1,"lbm_read_time_us":1356,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1182,"lbm_writes_lt_1ms":32,"mutex_wait_us":63,"num_input_rowsets":2,"peak_mem_usage":20738,"reinsert_count":0,"rows_written":141,"spinlock_wait_cycles":1408,"update_count":1}
I20260811 02:58:23.756445 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6247 bytes on disk
I20260811 02:58:23.757115 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.757697 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:23.765106 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.008s sys 0.001s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":82,"dirs.run_cpu_time_us":225,"dirs.run_wall_time_us":1235,"drs_written":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1464,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:23.766202 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.252800
I20260811 02:58:23.769757 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.011s sys 0.001s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":78,"dirs.run_cpu_time_us":245,"dirs.run_wall_time_us":1171,"drs_written":1,"lbm_read_time_us":56,"lbm_reads_lt_1ms":4,"lbm_write_time_us":787,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:23.770717 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.253266
I20260811 02:58:23.797003 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59616:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:23.807256 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.024s sys 0.008s Metrics: {"bytes_written":36350,"cfile_cache_hit":12,"cfile_cache_hit_bytes":27056,"cfile_cache_miss":28,"cfile_cache_miss_bytes":40278,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":857,"dirs.run_cpu_time_us":733,"dirs.run_wall_time_us":3388,"drs_written":1,"lbm_read_time_us":1651,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1742,"lbm_writes_lt_1ms":28,"mutex_wait_us":454,"num_input_rowsets":2,"peak_mem_usage":21001,"rows_written":141,"spinlock_wait_cycles":22784}
I20260811 02:58:23.809146 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6260 bytes on disk
I20260811 02:58:23.809854 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.811383 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260811 02:58:23.815482 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.049s user 0.036s sys 0.001s Metrics: {"bytes_written":36350,"cfile_cache_hit":13,"cfile_cache_hit_bytes":27053,"cfile_cache_miss":31,"cfile_cache_miss_bytes":40657,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1280,"dirs.run_cpu_time_us":943,"dirs.run_wall_time_us":7700,"drs_written":1,"lbm_read_time_us":1715,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1316,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":21054,"rows_written":141}
I20260811 02:58:23.816655 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 88a45d23cfdd4a6a9341932bb7e82b91 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.70 [id=52d8853b29b640c4988a84b4de683366]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:23.816746 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6260 bytes on disk
I20260811 02:58:23.817171 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a45d23cfdd4a6a9341932bb7e82b91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.818228 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.818979 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000015
I20260811 02:58:23.823864 20268 tablet_bootstrap.cc:492] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:23.825703 20268 tablet_bootstrap.cc:654] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.827018 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.009s sys 0.003s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":38,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1520,"drs_written":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1086,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:23.828058 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260811 02:58:23.829008 20268 tablet_bootstrap.cc:492] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:23.829200 20268 ts_tablet_manager.cc:1397] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:23.830127 20268 raft_consensus.cc:348] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.830359 20268 raft_consensus.cc:374] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.830482 20268 raft_consensus.cc:729] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.830775 20268 consensus_queue.cc:260] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.831012 20268 raft_consensus.cc:388] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.831111 20268 raft_consensus.cc:482] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.831239 20268 raft_consensus.cc:3037] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.831418 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.002s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":79,"dirs.run_cpu_time_us":159,"dirs.run_wall_time_us":1471,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":954,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:23.832597 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:23.833434 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.834090 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.199754
I20260811 02:58:23.834546 20268 raft_consensus.cc:504] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.834836 20268 leader_election.cc:302] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:23.835120 20272 raft_consensus.cc:2781] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.835258 20272 raft_consensus.cc:686] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:23.835516 20272 consensus_queue.cc:237] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.838351 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":178,"lbm_read_time_us":544,"lbm_reads_lt_1ms":25,"lbm_write_time_us":336,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:23.838987 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 167 bytes on disk
I20260811 02:58:23.842706 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.843523 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.199754
I20260811 02:58:23.844542 20268 leader_election.cc:288] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.845582 19970 catalog_manager.cc:5495] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:23.849877 20268 ts_tablet_manager.cc:1428] T 88a45d23cfdd4a6a9341932bb7e82b91 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20260811 02:58:23.866453 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.025s sys 0.003s Metrics: {"bytes_written":36741,"cfile_cache_hit":12,"cfile_cache_hit_bytes":27398,"cfile_cache_miss":28,"cfile_cache_miss_bytes":40588,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":351,"dirs.run_cpu_time_us":899,"dirs.run_wall_time_us":5453,"drs_written":1,"lbm_read_time_us":916,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1269,"lbm_writes_lt_1ms":28,"mutex_wait_us":3,"num_input_rowsets":2,"peak_mem_usage":21088,"rows_written":143}
I20260811 02:58:23.868563 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6334 bytes on disk
I20260811 02:58:23.870254 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.871816 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000004
I20260811 02:58:23.879669 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.003s sys 0.000s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":194,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:23.880322 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260811 02:58:23.886111 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.003s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":242,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:23.886826 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260811 02:58:23.887718 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.031s sys 0.000s Metrics: {"bytes_written":390,"cfile_cache_hit":13,"cfile_cache_hit_bytes":27431,"cfile_cache_miss":31,"cfile_cache_miss_bytes":40657,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1200,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":1292,"drs_written":1,"lbm_read_time_us":1296,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1404,"lbm_writes_lt_1ms":32,"mutex_wait_us":2,"num_input_rowsets":2,"peak_mem_usage":21105,"reinsert_count":0,"rows_written":143,"update_count":2}
I20260811 02:58:23.888597 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6334 bytes on disk
I20260811 02:58:23.889223 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4}
I20260811 02:58:23.895690 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.005s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":104,"lbm_read_time_us":971,"lbm_reads_lt_1ms":12,"lbm_write_time_us":189,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260811 02:58:23.920399 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59620:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:23.935014 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 450728d432b849aba142775b32721d24 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.71 [id=981d8a6fc11346c68fd23a8c63076a5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:23.935505 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 450728d432b849aba142775b32721d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.941246 20268 tablet_bootstrap.cc:492] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:23.942814 20268 tablet_bootstrap.cc:654] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.945366 20268 tablet_bootstrap.cc:492] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:23.945698 20268 ts_tablet_manager.cc:1397] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:23.946520 20268 raft_consensus.cc:348] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.946719 20268 raft_consensus.cc:374] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.946820 20268 raft_consensus.cc:729] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.947090 20268 consensus_queue.cc:260] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.947275 20268 raft_consensus.cc:388] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.947403 20268 raft_consensus.cc:482] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.947510 20268 raft_consensus.cc:3037] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.950035 20268 raft_consensus.cc:504] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.950301 20268 leader_election.cc:302] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:23.950548 20268 leader_election.cc:288] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.950626 20272 raft_consensus.cc:2781] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.951206 20272 raft_consensus.cc:686] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:23.951360 20268 ts_tablet_manager.cc:1428] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:58:23.951509 20272 consensus_queue.cc:237] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:23.958271 19968 catalog_manager.cc:5495] T 450728d432b849aba142775b32721d24 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.001888 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59628:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:24.011945 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260811 02:58:24.016413 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 905f7a97266c4035a4d11a2ba6a2ae27 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.72 [id=9131beef77234ce593d25f49fed62ea8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:24.016928 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 905f7a97266c4035a4d11a2ba6a2ae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:24.023546 20268 tablet_bootstrap.cc:492] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:24.025535 20268 tablet_bootstrap.cc:654] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:24.026589 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.008s sys 0.004s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":46,"dirs.run_cpu_time_us":230,"dirs.run_wall_time_us":1376,"drs_written":1,"lbm_read_time_us":56,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1083,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:24.027765 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.245353
I20260811 02:58:24.028239 20268 tablet_bootstrap.cc:492] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:24.028424 20268 ts_tablet_manager.cc:1397] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:24.029371 20268 raft_consensus.cc:348] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.029650 20268 raft_consensus.cc:374] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:24.029776 20268 raft_consensus.cc:729] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:24.030138 20268 consensus_queue.cc:260] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.030417 20268 raft_consensus.cc:388] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:24.030524 20268 raft_consensus.cc:482] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:24.030654 20268 raft_consensus.cc:3037] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:24.033960 20268 raft_consensus.cc:504] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.034337 20268 leader_election.cc:302] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:24.034595 20268 leader_election.cc:288] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:24.034870 20272 raft_consensus.cc:2781] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:24.035065 20272 raft_consensus.cc:686] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:24.035218 20268 ts_tablet_manager.cc:1428] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260811 02:58:24.035337 20272 consensus_queue.cc:237] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.038908 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.006s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1062,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":172,"lbm_read_time_us":654,"lbm_reads_lt_1ms":25,"lbm_write_time_us":363,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:24.040354 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 487 bytes on disk
I20260811 02:58:24.041905 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.041816 19968 catalog_manager.cc:5495] T 905f7a97266c4035a4d11a2ba6a2ae27 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.042763 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.220741
I20260811 02:58:24.074045 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.025s sys 0.005s Metrics: {"bytes_written":337,"cfile_cache_hit":14,"cfile_cache_hit_bytes":28164,"cfile_cache_miss":34,"cfile_cache_miss_bytes":42392,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":8,"dirs.queue_time_us":1016,"dirs.run_cpu_time_us":496,"dirs.run_wall_time_us":2692,"drs_written":1,"lbm_read_time_us":1294,"lbm_reads_lt_1ms":70,"lbm_write_time_us":1119,"lbm_writes_lt_1ms":32,"mutex_wait_us":238,"num_input_rowsets":2,"peak_mem_usage":21872,"reinsert_count":0,"rows_written":147,"spinlock_wait_cycles":27392,"update_count":1}
I20260811 02:58:24.075237 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6512 bytes on disk
I20260811 02:58:24.076164 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.089083 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59632:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:24.102566 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 903e0881c69744bbb23ce04873063ada (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.73 [id=6d6304aa4fd747b1bf2d86464ce31bb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:24.103065 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903e0881c69744bbb23ce04873063ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:24.107748 20268 tablet_bootstrap.cc:492] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:24.109100 20268 tablet_bootstrap.cc:654] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:24.111617 20268 tablet_bootstrap.cc:492] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:24.111907 20268 ts_tablet_manager.cc:1397] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:24.113009 20268 raft_consensus.cc:348] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.113250 20268 raft_consensus.cc:374] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:24.113335 20268 raft_consensus.cc:729] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:24.113584 20268 consensus_queue.cc:260] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.113806 20268 raft_consensus.cc:388] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:24.113883 20268 raft_consensus.cc:482] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:24.113953 20268 raft_consensus.cc:3037] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:24.115788 20268 raft_consensus.cc:504] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.116007 20268 leader_election.cc:302] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:24.116218 20268 leader_election.cc:288] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:24.116266 20272 raft_consensus.cc:2781] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:24.116458 20272 raft_consensus.cc:686] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:24.116755 20268 ts_tablet_manager.cc:1428] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:24.116729 20272 consensus_queue.cc:237] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.123253 19970 catalog_manager.cc:5495] T 903e0881c69744bbb23ce04873063ada P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.140249 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260811 02:58:24.146930 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000088
I20260811 02:58:24.151352 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.005s sys 0.004s Metrics: {"bytes_written":6066,"cfile_init":1,"dirs.queue_time_us":116,"dirs.run_cpu_time_us":215,"dirs.run_wall_time_us":1418,"drs_written":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4,"lbm_write_time_us":852,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:24.152220 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.220428
I20260811 02:58:24.159274 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.010s sys 0.000s Metrics: {"bytes_written":6066,"cfile_init":1,"dirs.queue_time_us":96,"dirs.run_cpu_time_us":257,"dirs.run_wall_time_us":1403,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":916,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:24.160068 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 518 bytes on disk
I20260811 02:58:24.160625 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.161197 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.220428
I20260811 02:58:24.191076 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59642:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:24.192085 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.022s sys 0.012s Metrics: {"bytes_written":38093,"cfile_cache_hit":13,"cfile_cache_hit_bytes":28533,"cfile_cache_miss":31,"cfile_cache_miss_bytes":42943,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":408,"dirs.run_cpu_time_us":236,"dirs.run_wall_time_us":1348,"drs_written":1,"lbm_read_time_us":1290,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1282,"lbm_writes_lt_1ms":28,"mutex_wait_us":22,"num_input_rowsets":2,"peak_mem_usage":22282,"rows_written":149}
I20260811 02:58:24.193073 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6608 bytes on disk
I20260811 02:58:24.193722 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.209100 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 58d4ce23f6a243b9bbe2feeae4496200 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.74 [id=732718d14a9347068432bff9cb599203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:24.209585 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58d4ce23f6a243b9bbe2feeae4496200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:24.209995 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.049s user 0.037s sys 0.002s Metrics: {"bytes_written":38093,"cfile_cache_hit":13,"cfile_cache_hit_bytes":28533,"cfile_cache_miss":31,"cfile_cache_miss_bytes":42943,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":953,"dirs.run_cpu_time_us":1047,"dirs.run_wall_time_us":8141,"drs_written":1,"lbm_read_time_us":3762,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":62,"lbm_write_time_us":1409,"lbm_writes_lt_1ms":28,"mutex_wait_us":291,"num_input_rowsets":2,"peak_mem_usage":22282,"rows_written":149,"thread_start_us":2719,"threads_started":1}
I20260811 02:58:24.211027 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6608 bytes on disk
I20260811 02:58:24.211745 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.212374 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000015
I20260811 02:58:24.218374 20268 tablet_bootstrap.cc:492] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:24.220364 20268 tablet_bootstrap.cc:654] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:24.223647 20268 tablet_bootstrap.cc:492] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:24.223826 20268 ts_tablet_manager.cc:1397] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260811 02:58:24.224610 20268 raft_consensus.cc:348] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.224817 20268 raft_consensus.cc:374] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:24.224900 20268 raft_consensus.cc:729] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:24.225116 20268 consensus_queue.cc:260] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.225353 20268 raft_consensus.cc:388] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:24.225446 20268 raft_consensus.cc:482] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:24.225561 20268 raft_consensus.cc:3037] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:24.231506 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.011s sys 0.000s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":204,"dirs.run_wall_time_us":4584,"drs_written":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1012,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:24.232522 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 167 bytes on disk
I20260811 02:58:24.233289 20268 raft_consensus.cc:504] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.233601 20268 leader_election.cc:302] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:24.233839 20268 leader_election.cc:288] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:24.233958 20272 raft_consensus.cc:2781] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:24.234493 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.235225 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260811 02:58:24.235607 20272 raft_consensus.cc:686] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:24.235888 20272 consensus_queue.cc:237] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.241328 20268 ts_tablet_manager.cc:1428] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.017s user 0.004s sys 0.002s
I20260811 02:58:24.244061 19970 catalog_manager.cc:5495] T 58d4ce23f6a243b9bbe2feeae4496200 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.246079 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":170,"lbm_read_time_us":413,"lbm_reads_lt_1ms":21,"lbm_write_time_us":339,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:24.246762 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.201547
I20260811 02:58:24.277491 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.026s sys 0.004s Metrics: {"bytes_written":334,"cfile_cache_hit":13,"cfile_cache_hit_bytes":28911,"cfile_cache_miss":31,"cfile_cache_miss_bytes":42833,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1413,"dirs.run_cpu_time_us":214,"dirs.run_wall_time_us":1742,"drs_written":1,"lbm_read_time_us":1364,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1022,"lbm_writes_lt_1ms":32,"mutex_wait_us":62,"num_input_rowsets":2,"peak_mem_usage":22223,"reinsert_count":0,"rows_written":151,"spinlock_wait_cycles":2304,"thread_start_us":270,"threads_started":1,"update_count":1}
I20260811 02:58:24.278484 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260811 02:58:24.284273 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:24.284942 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6676 bytes on disk
I20260811 02:58:24.285734 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.286386 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260811 02:58:24.292078 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":86,"lbm_read_time_us":184,"lbm_reads_lt_1ms":12,"lbm_write_time_us":154,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260811 02:58:24.314076 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59654:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:24.327373 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260811 02:58:24.334882 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 2a8dcc97cc154a8f9049ef68a1eb0c4a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.75 [id=219e8cded8ea4aa3a84fba63167be2f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:24.335433 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a8dcc97cc154a8f9049ef68a1eb0c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:24.342491 20268 tablet_bootstrap.cc:492] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:24.342854 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.013s sys 0.000s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":74,"dirs.run_cpu_time_us":225,"dirs.run_wall_time_us":1321,"drs_written":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4,"lbm_write_time_us":892,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:24.343863 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.245353
I20260811 02:58:24.344270 20268 tablet_bootstrap.cc:654] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:24.346920 20268 tablet_bootstrap.cc:492] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:24.347194 20268 ts_tablet_manager.cc:1397] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260811 02:58:24.348143 20268 raft_consensus.cc:348] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.348423 20268 raft_consensus.cc:374] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:24.348527 20268 raft_consensus.cc:729] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:24.348803 20268 consensus_queue.cc:260] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.349048 20268 raft_consensus.cc:388] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:24.349176 20268 raft_consensus.cc:482] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:24.349290 20268 raft_consensus.cc:3037] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:24.352330 20268 raft_consensus.cc:504] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.352622 20268 leader_election.cc:302] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:24.352883 20272 raft_consensus.cc:2781] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:24.353053 20272 raft_consensus.cc:686] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:24.353315 20272 consensus_queue.cc:237] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.354410 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1062,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":110,"lbm_read_time_us":566,"lbm_reads_lt_1ms":25,"lbm_write_time_us":369,"lbm_writes_lt_1ms":9,"mutex_wait_us":25,"reinsert_count":0,"update_count":1}
I20260811 02:58:24.355068 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.226652
I20260811 02:58:24.359895 19970 catalog_manager.cc:5495] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.367951 20268 leader_election.cc:288] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:24.380435 20268 ts_tablet_manager.cc:1428] T 2a8dcc97cc154a8f9049ef68a1eb0c4a P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.033s user 0.005s sys 0.001s
I20260811 02:58:24.405990 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.051s user 0.035s sys 0.000s Metrics: {"bytes_written":390,"cfile_cache_hit":14,"cfile_cache_hit_bytes":29269,"cfile_cache_miss":34,"cfile_cache_miss_bytes":44013,"cfile_init":10,"delete_count":0,"delta_iterators_relevant":8,"dirs.queue_time_us":846,"dirs.run_cpu_time_us":725,"dirs.run_wall_time_us":4230,"drs_written":1,"lbm_read_time_us":1629,"lbm_reads_lt_1ms":74,"lbm_write_time_us":1500,"lbm_writes_lt_1ms":32,"mutex_wait_us":118,"num_input_rowsets":2,"peak_mem_usage":22845,"reinsert_count":0,"rows_written":153,"spinlock_wait_cycles":1792,"update_count":2}
I20260811 02:58:24.407016 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6768 bytes on disk
I20260811 02:58:24.407609 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.431943 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59666:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:24.445657 20165 tablet_service.cc:1467] Processing CreateTablet for tablet d21675ee5e854963b0fdf575a11bed75 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.76 [id=512996742de64a34ba2b413b3e09db7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:24.445919 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260811 02:58:24.446229 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21675ee5e854963b0fdf575a11bed75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:24.451941 20268 tablet_bootstrap.cc:492] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:24.453738 20268 tablet_bootstrap.cc:654] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:24.456128 20268 tablet_bootstrap.cc:492] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:24.456295 20268 ts_tablet_manager.cc:1397] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:24.457027 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.007s sys 0.000s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":225,"dirs.run_wall_time_us":1370,"drs_written":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4,"lbm_write_time_us":800,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:24.457222 20268 raft_consensus.cc:348] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.457451 20268 raft_consensus.cc:374] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:24.457556 20268 raft_consensus.cc:729] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:24.457921 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.246697
I20260811 02:58:24.457845 20268 consensus_queue.cc:260] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.458105 20268 raft_consensus.cc:388] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:24.458206 20268 raft_consensus.cc:482] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:24.458333 20268 raft_consensus.cc:3037] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:24.460289 20268 raft_consensus.cc:504] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.460584 20268 leader_election.cc:302] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:24.460829 20268 leader_election.cc:288] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:24.460952 20272 raft_consensus.cc:2781] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:24.461148 20272 raft_consensus.cc:686] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:24.461406 20272 consensus_queue.cc:237] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.461872 20268 ts_tablet_manager.cc:1428] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260811 02:58:24.474042 19968 catalog_manager.cc:5495] T d21675ee5e854963b0fdf575a11bed75 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.494374 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.021s sys 0.012s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":29614,"cfile_cache_miss":28,"cfile_cache_miss_bytes":44122,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":877,"dirs.run_cpu_time_us":215,"dirs.run_wall_time_us":3362,"drs_written":1,"lbm_read_time_us":1596,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1263,"lbm_writes_lt_1ms":32,"mutex_wait_us":77,"num_input_rowsets":2,"peak_mem_usage":23242,"reinsert_count":0,"rows_written":155,"spinlock_wait_cycles":4352,"update_count":1}
I20260811 02:58:24.495489 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6868 bytes on disk
I20260811 02:58:24.496071 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.519315 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59674:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.77"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:24.534139 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 8b6f293573de4571a7fd8870c769912b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.77 [id=8adbfa0294f345e8a5efedc92f3f92f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:24.534631 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b6f293573de4571a7fd8870c769912b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:24.539120 20268 tablet_bootstrap.cc:492] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:24.540488 20268 tablet_bootstrap.cc:654] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:24.542768 20268 tablet_bootstrap.cc:492] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:24.542984 20268 ts_tablet_manager.cc:1397] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:24.543179 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260811 02:58:24.543733 20268 raft_consensus.cc:348] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.543960 20268 raft_consensus.cc:374] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:24.544076 20268 raft_consensus.cc:729] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:24.544322 20268 consensus_queue.cc:260] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.544513 20268 raft_consensus.cc:388] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:24.544634 20268 raft_consensus.cc:482] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:24.544754 20268 raft_consensus.cc:3037] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:24.547068 20268 raft_consensus.cc:504] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.547353 20268 leader_election.cc:302] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:24.547638 20268 leader_election.cc:288] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:24.547670 20272 raft_consensus.cc:2781] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:24.547859 20272 raft_consensus.cc:686] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:24.548107 20272 consensus_queue.cc:237] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.548290 20268 ts_tablet_manager.cc:1428] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 02:58:24.556504 19968 catalog_manager.cc:5495] T 8b6f293573de4571a7fd8870c769912b P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.558452 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.009s sys 0.003s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":64,"dirs.run_cpu_time_us":261,"dirs.run_wall_time_us":1196,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":911,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6,"spinlock_wait_cycles":5504}
I20260811 02:58:24.562503 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 528 bytes on disk
I20260811 02:58:24.563817 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.564668 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.245795
I20260811 02:58:24.593097 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.026s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":29976,"cfile_cache_miss":31,"cfile_cache_miss_bytes":45090,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":894,"dirs.run_cpu_time_us":718,"dirs.run_wall_time_us":3239,"drs_written":1,"lbm_read_time_us":1243,"lbm_reads_lt_1ms":63,"lbm_write_time_us":998,"lbm_writes_lt_1ms":32,"mutex_wait_us":128,"num_input_rowsets":2,"peak_mem_usage":23545,"reinsert_count":0,"rows_written":157,"update_count":1}
I20260811 02:58:24.594573 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6955 bytes on disk
I20260811 02:58:24.595615 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.605556 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59688:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:24.618835 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 825d78e881cf4862bb4b927330d2f5dc (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.78 [id=b8f7faa803f24dab8ace9da7c8c42d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:24.619311 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 825d78e881cf4862bb4b927330d2f5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:24.624485 20268 tablet_bootstrap.cc:492] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:24.626037 20268 tablet_bootstrap.cc:654] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:24.628481 20268 tablet_bootstrap.cc:492] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:24.628626 20268 ts_tablet_manager.cc:1397] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:24.629264 20268 raft_consensus.cc:348] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.629429 20268 raft_consensus.cc:374] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:24.629503 20268 raft_consensus.cc:729] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:24.629768 20268 consensus_queue.cc:260] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.629962 20268 raft_consensus.cc:388] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:24.630025 20268 raft_consensus.cc:482] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:24.630110 20268 raft_consensus.cc:3037] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:24.632195 20268 raft_consensus.cc:504] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.632450 20268 leader_election.cc:302] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:24.632705 20268 leader_election.cc:288] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:24.632875 20272 raft_consensus.cc:2781] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:24.633144 20272 raft_consensus.cc:686] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:24.633314 20268 ts_tablet_manager.cc:1428] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260811 02:58:24.633446 20272 consensus_queue.cc:237] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.639634 19970 catalog_manager.cc:5495] T 825d78e881cf4862bb4b927330d2f5dc P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.658593 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000088
I20260811 02:58:24.669086 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.007s sys 0.000s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":69,"dirs.run_cpu_time_us":244,"dirs.run_wall_time_us":1473,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":889,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:24.670001 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 523 bytes on disk
I20260811 02:58:24.670608 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.671422 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.244442
I20260811 02:58:24.690824 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59690:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:24.706351 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 26cb8235716f42a5afe0baa08493a372 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.79 [id=fbf79c38425748fea1a87b7f6d04baf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:24.706887 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26cb8235716f42a5afe0baa08493a372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:24.706867 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.025s sys 0.007s Metrics: {"bytes_written":40272,"cfile_cache_hit":13,"cfile_cache_hit_bytes":30388,"cfile_cache_miss":31,"cfile_cache_miss_bytes":45684,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":596,"dirs.run_cpu_time_us":244,"dirs.run_wall_time_us":1420,"drs_written":1,"lbm_read_time_us":1339,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1163,"lbm_writes_lt_1ms":28,"mutex_wait_us":76,"num_input_rowsets":2,"peak_mem_usage":23972,"rows_written":159,"spinlock_wait_cycles":768}
I20260811 02:58:24.707990 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7055 bytes on disk
I20260811 02:58:24.708585 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.709165 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260811 02:58:24.712787 20268 tablet_bootstrap.cc:492] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:24.714699 20268 tablet_bootstrap.cc:654] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:24.717092 20268 tablet_bootstrap.cc:492] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:24.717269 20268 ts_tablet_manager.cc:1397] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 02:58:24.718097 20268 raft_consensus.cc:348] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.718367 20268 raft_consensus.cc:374] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:24.718461 20268 raft_consensus.cc:729] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:24.718709 20268 consensus_queue.cc:260] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.718925 20268 raft_consensus.cc:388] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:24.719048 20268 raft_consensus.cc:482] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:24.719139 20268 raft_consensus.cc:3037] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:24.720980 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":1012,"drs_written":1,"lbm_read_time_us":108,"lbm_reads_lt_1ms":4,"lbm_write_time_us":787,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:24.721917 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.163491
I20260811 02:58:24.722750 20268 raft_consensus.cc:504] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.723079 20268 leader_election.cc:302] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:24.723312 20268 leader_election.cc:288] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:24.723362 20272 raft_consensus.cc:2781] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:24.723620 20272 raft_consensus.cc:686] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:24.723879 20272 consensus_queue.cc:237] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.724794 20268 ts_tablet_manager.cc:1428] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260811 02:58:24.742162 19970 catalog_manager.cc:5495] T 26cb8235716f42a5afe0baa08493a372 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.747064 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:24.761317 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.010s sys 0.000s Metrics: {"bytes_written":334,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":58,"dirs.run_cpu_time_us":272,"dirs.run_wall_time_us":1478,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1149,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:24.762300 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.228924
I20260811 02:58:24.765125 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.030s sys 0.001s Metrics: {"bytes_written":40693,"cfile_cache_hit":12,"cfile_cache_hit_bytes":30723,"cfile_cache_miss":28,"cfile_cache_miss_bytes":45507,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":364,"dirs.run_cpu_time_us":228,"dirs.run_wall_time_us":1212,"drs_written":1,"lbm_read_time_us":1316,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1125,"lbm_writes_lt_1ms":28,"mutex_wait_us":164,"num_input_rowsets":2,"peak_mem_usage":23918,"rows_written":161,"spinlock_wait_cycles":14208}
I20260811 02:58:24.766575 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7125 bytes on disk
I20260811 02:58:24.767632 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.768230 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000005
I20260811 02:58:24.773936 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.005s sys 0.000s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":170,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:24.775086 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260811 02:58:24.778035 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.009s sys 0.001s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1180,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":102,"lbm_read_time_us":543,"lbm_reads_lt_1ms":25,"lbm_write_time_us":2083,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":8,"reinsert_count":0,"update_count":1}
I20260811 02:58:24.778782 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.223267
I20260811 02:58:24.780014 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":145,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:24.780680 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260811 02:58:24.787076 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.005s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":63,"lbm_read_time_us":164,"lbm_reads_lt_1ms":12,"lbm_write_time_us":229,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260811 02:58:24.815398 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59698:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:24.819027 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.032s sys 0.000s Metrics: {"bytes_written":40730,"cfile_cache_hit":14,"cfile_cache_hit_bytes":30784,"cfile_cache_miss":34,"cfile_cache_miss_bytes":46536,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":2772,"dirs.run_cpu_time_us":660,"dirs.run_wall_time_us":3574,"drs_written":1,"lbm_read_time_us":1534,"lbm_reads_lt_1ms":74,"lbm_write_time_us":1282,"lbm_writes_lt_1ms":28,"mutex_wait_us":19,"num_input_rowsets":2,"peak_mem_usage":24249,"rows_written":161,"thread_start_us":390,"threads_started":1}
I20260811 02:58:24.820513 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7150 bytes on disk
I20260811 02:58:24.821228 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.829169 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 857810a11c53494385079ed756e6200d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.80 [id=bffad54cd02b4747aa106a485be82300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:24.829777 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 857810a11c53494385079ed756e6200d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:24.834240 20268 tablet_bootstrap.cc:492] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:24.835647 20268 tablet_bootstrap.cc:654] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:24.837949 20268 tablet_bootstrap.cc:492] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:24.838131 20268 ts_tablet_manager.cc:1397] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:24.838821 20268 raft_consensus.cc:348] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.839023 20268 raft_consensus.cc:374] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:24.839143 20268 raft_consensus.cc:729] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:24.839388 20268 consensus_queue.cc:260] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.839599 20268 raft_consensus.cc:388] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:24.839685 20268 raft_consensus.cc:482] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:24.839838 20268 raft_consensus.cc:3037] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:24.842064 20268 raft_consensus.cc:504] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.842358 20268 leader_election.cc:302] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:24.842563 20268 leader_election.cc:288] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:24.842739 20272 raft_consensus.cc:2781] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:24.842937 20272 raft_consensus.cc:686] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:24.843109 20268 ts_tablet_manager.cc:1428] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:24.843184 20272 consensus_queue.cc:237] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.846873 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:24.849327 19970 catalog_manager.cc:5495] T 857810a11c53494385079ed756e6200d P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.866349 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.010s sys 0.003s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":50,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":1248,"drs_written":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1228,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:24.867229 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.230980
I20260811 02:58:24.876102 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1183,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":162,"lbm_read_time_us":428,"lbm_reads_lt_1ms":25,"lbm_write_time_us":256,"lbm_writes_lt_1ms":9,"mutex_wait_us":34,"reinsert_count":0,"update_count":1}
I20260811 02:58:24.876798 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 531 bytes on disk
I20260811 02:58:24.877415 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.878294 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.190150
I20260811 02:58:24.906319 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59704:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:24.918758 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.037s sys 0.000s Metrics: {"bytes_written":41157,"cfile_cache_hit":14,"cfile_cache_hit_bytes":31162,"cfile_cache_miss":34,"cfile_cache_miss_bytes":47128,"cfile_init":9,"delta_iterators_relevant":8,"dirs.queue_time_us":1842,"dirs.run_cpu_time_us":597,"dirs.run_wall_time_us":3369,"drs_written":1,"lbm_read_time_us":1632,"lbm_reads_lt_1ms":70,"lbm_write_time_us":948,"lbm_writes_lt_1ms":28,"mutex_wait_us":530,"num_input_rowsets":2,"peak_mem_usage":24711,"rows_written":163,"spinlock_wait_cycles":3840}
I20260811 02:58:24.919890 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000015
I20260811 02:58:24.923156 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 21835ae79d3a48e0b83d567a19e67607 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.81 [id=4a807a4680e04074b18633e1605fdfa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:24.923676 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21835ae79d3a48e0b83d567a19e67607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:24.930411 20268 tablet_bootstrap.cc:492] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:24.931943 20268 tablet_bootstrap.cc:654] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:24.933511 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.000s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":44,"dirs.run_cpu_time_us":163,"dirs.run_wall_time_us":1155,"drs_written":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1087,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:24.934589 20268 tablet_bootstrap.cc:492] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:24.934659 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7402 bytes on disk
I20260811 02:58:24.934896 20268 ts_tablet_manager.cc:1397] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:58:24.935417 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":136,"lbm_reads_lt_1ms":8}
I20260811 02:58:24.935940 20268 raft_consensus.cc:348] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.936051 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260811 02:58:24.936179 20268 raft_consensus.cc:374] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:24.936283 20268 raft_consensus.cc:729] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:24.936529 20268 consensus_queue.cc:260] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.936731 20268 raft_consensus.cc:388] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:24.936863 20268 raft_consensus.cc:482] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:24.936991 20268 raft_consensus.cc:3037] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:24.939544 20268 raft_consensus.cc:504] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.939816 20268 leader_election.cc:302] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:24.940075 20268 leader_election.cc:288] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:24.940167 20272 raft_consensus.cc:2781] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:24.940382 20272 raft_consensus.cc:686] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:24.940641 20268 ts_tablet_manager.cc:1428] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:58:24.940620 20272 consensus_queue.cc:237] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:24.947495 19968 catalog_manager.cc:5495] T 21835ae79d3a48e0b83d567a19e67607 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:24.949955 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":324,"lbm_read_time_us":721,"lbm_reads_lt_1ms":21,"lbm_write_time_us":285,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:24.951723 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.199292
I20260811 02:58:24.983570 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.025s sys 0.003s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":31496,"cfile_cache_miss":31,"cfile_cache_miss_bytes":46690,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":2669,"dirs.run_cpu_time_us":1208,"dirs.run_wall_time_us":7128,"drs_written":1,"lbm_read_time_us":1133,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1054,"lbm_writes_lt_1ms":32,"mutex_wait_us":166,"num_input_rowsets":2,"peak_mem_usage":24635,"reinsert_count":0,"rows_written":165,"spinlock_wait_cycles":6784,"thread_start_us":1065,"threads_started":1,"update_count":1}
I20260811 02:58:24.985980 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7303 bytes on disk
I20260811 02:58:24.986825 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":170,"lbm_reads_lt_1ms":4}
I20260811 02:58:24.999652 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59710:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:25.014343 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 00bba18758614d92ba91c16024665989 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.82 [id=5d01b3dcaa0c47418e16269dd2bc07a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:25.014868 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00bba18758614d92ba91c16024665989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.019737 20268 tablet_bootstrap.cc:492] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:25.021101 20268 tablet_bootstrap.cc:654] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.023561 20268 tablet_bootstrap.cc:492] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:25.023736 20268 ts_tablet_manager.cc:1397] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:25.024370 20268 raft_consensus.cc:348] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.024582 20268 raft_consensus.cc:374] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.024713 20268 raft_consensus.cc:729] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.024948 20268 consensus_queue.cc:260] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.025156 20268 raft_consensus.cc:388] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.025259 20268 raft_consensus.cc:482] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.025394 20268 raft_consensus.cc:3037] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.027634 20268 raft_consensus.cc:504] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.027906 20268 leader_election.cc:302] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:25.028151 20268 leader_election.cc:288] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.028203 20272 raft_consensus.cc:2781] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.028425 20272 raft_consensus.cc:686] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:25.028721 20268 ts_tablet_manager.cc:1428] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260811 02:58:25.028749 20272 consensus_queue.cc:237] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.035444 19968 catalog_manager.cc:5495] T 00bba18758614d92ba91c16024665989 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:25.038220 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000088
I20260811 02:58:25.055027 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.005s sys 0.007s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":50,"dirs.run_cpu_time_us":235,"dirs.run_wall_time_us":1413,"drs_written":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1227,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:25.056023 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.230980
I20260811 02:58:25.065019 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1183,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":205,"lbm_read_time_us":498,"lbm_reads_lt_1ms":25,"lbm_write_time_us":363,"lbm_writes_lt_1ms":9,"mutex_wait_us":1,"reinsert_count":0,"update_count":1}
I20260811 02:58:25.065843 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.223946
I20260811 02:58:25.072353 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000086
I20260811 02:58:25.086035 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":262,"dirs.run_wall_time_us":1432,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":942,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:25.087193 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 517 bytes on disk
I20260811 02:58:25.088316 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.090342 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.217143
I20260811 02:58:25.100600 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59712:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:25.102880 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.030s sys 0.004s Metrics: {"bytes_written":42029,"cfile_cache_hit":14,"cfile_cache_hit_bytes":31902,"cfile_cache_miss":34,"cfile_cache_miss_bytes":48288,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":1575,"dirs.run_cpu_time_us":1185,"dirs.run_wall_time_us":8885,"drs_written":1,"lbm_read_time_us":1286,"lbm_reads_lt_1ms":74,"lbm_write_time_us":1098,"lbm_writes_lt_1ms":28,"mutex_wait_us":100,"num_input_rowsets":2,"peak_mem_usage":25221,"rows_written":167,"spinlock_wait_cycles":14080,"thread_start_us":2407,"threads_started":1}
I20260811 02:58:25.103977 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7394 bytes on disk
I20260811 02:58:25.104679 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.117283 20165 tablet_service.cc:1467] Processing CreateTablet for tablet f86a44d625444a1d82ee58eff9134c13 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.83 [id=7cc7f3affb054ff1898ca12e41b97942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:25.117883 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86a44d625444a1d82ee58eff9134c13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.124580 20268 tablet_bootstrap.cc:492] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:25.126353 20268 tablet_bootstrap.cc:654] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.129356 20268 tablet_bootstrap.cc:492] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:25.129558 20268 ts_tablet_manager.cc:1397] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:25.130331 20268 raft_consensus.cc:348] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.130613 20268 raft_consensus.cc:374] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.130735 20268 raft_consensus.cc:729] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.131078 20268 consensus_queue.cc:260] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.131321 20268 raft_consensus.cc:388] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.131428 20268 raft_consensus.cc:482] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.131551 20268 raft_consensus.cc:3037] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.132647 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.042s user 0.030s sys 0.001s Metrics: {"bytes_written":42029,"cfile_cache_hit":12,"cfile_cache_hit_bytes":31871,"cfile_cache_miss":28,"cfile_cache_miss_bytes":47429,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":661,"dirs.run_cpu_time_us":267,"dirs.run_wall_time_us":1440,"drs_written":1,"lbm_read_time_us":1438,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1238,"lbm_writes_lt_1ms":28,"mutex_wait_us":101,"num_input_rowsets":2,"peak_mem_usage":25154,"rows_written":167,"spinlock_wait_cycles":1408}
I20260811 02:58:25.133430 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7394 bytes on disk
I20260811 02:58:25.134068 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":74,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.134234 20268 raft_consensus.cc:504] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.134558 20268 leader_election.cc:302] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:25.134645 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000014
I20260811 02:58:25.134846 20268 leader_election.cc:288] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.134891 20272 raft_consensus.cc:2781] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.135102 20272 raft_consensus.cc:686] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:25.135483 20268 ts_tablet_manager.cc:1428] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260811 02:58:25.135371 20272 consensus_queue.cc:237] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.141542 19968 catalog_manager.cc:5495] T f86a44d625444a1d82ee58eff9134c13 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:25.159713 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.008s sys 0.004s Metrics: {"bytes_written":335,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":1219,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1845,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"spinlock_wait_cycles":896,"update_count":1}
I20260811 02:58:25.160844 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:25.161548 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.162348 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.463347
I20260811 02:58:25.189770 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.023s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":579,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":124,"lbm_read_time_us":334,"lbm_reads_lt_1ms":21,"lbm_write_time_us":265,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:25.190438 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.231569
I20260811 02:58:25.197166 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59724:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:25.217267 20165 tablet_service.cc:1467] Processing CreateTablet for tablet b76d641b437e456d8b86cad3bc154731 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.84 [id=4c9980eb836041d9b6187ae7d8ead903]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:25.217828 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b76d641b437e456d8b86cad3bc154731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.220567 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.023s sys 0.004s Metrics: {"bytes_written":42474,"cfile_cache_hit":13,"cfile_cache_hit_bytes":32274,"cfile_cache_miss":31,"cfile_cache_miss_bytes":48146,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":962,"dirs.run_cpu_time_us":221,"dirs.run_wall_time_us":1182,"drs_written":1,"lbm_read_time_us":1084,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1015,"lbm_writes_lt_1ms":28,"mutex_wait_us":58,"num_input_rowsets":2,"peak_mem_usage":25428,"rows_written":169,"spinlock_wait_cycles":20608}
I20260811 02:58:25.225687 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7475 bytes on disk
I20260811 02:58:25.225813 20268 tablet_bootstrap.cc:492] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:25.226418 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.227029 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000019
I20260811 02:58:25.227705 20268 tablet_bootstrap.cc:654] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.236088 20268 tablet_bootstrap.cc:492] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:25.236302 20268 ts_tablet_manager.cc:1397] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 02:58:25.237459 20268 raft_consensus.cc:348] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.237876 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260811 02:58:25.238937 20268 raft_consensus.cc:374] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.239089 20268 raft_consensus.cc:729] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.239147 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":55,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1380,"drs_written":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4,"lbm_write_time_us":819,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:25.239341 20268 consensus_queue.cc:260] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.239693 20268 raft_consensus.cc:388] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.239825 20268 raft_consensus.cc:482] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.239987 20268 raft_consensus.cc:3037] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.240092 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.222760
I20260811 02:58:25.243125 20268 raft_consensus.cc:504] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.243503 20268 leader_election.cc:302] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:25.243778 20268 leader_election.cc:288] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.243876 20272 raft_consensus.cc:2781] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.244031 20272 raft_consensus.cc:686] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:25.244261 20272 consensus_queue.cc:237] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.245749 20268 ts_tablet_manager.cc:1428] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260811 02:58:25.257011 19968 catalog_manager.cc:5495] T b76d641b437e456d8b86cad3bc154731 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:25.266556 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.009s sys 0.003s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":118,"dirs.run_cpu_time_us":246,"dirs.run_wall_time_us":1214,"drs_written":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4,"lbm_write_time_us":938,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:25.267659 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 506 bytes on disk
I20260811 02:58:25.268695 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.269577 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242795
I20260811 02:58:25.273371 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.024s sys 0.005s Metrics: {"bytes_written":42871,"cfile_cache_hit":12,"cfile_cache_hit_bytes":32578,"cfile_cache_miss":28,"cfile_cache_miss_bytes":48238,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":597,"dirs.run_cpu_time_us":980,"dirs.run_wall_time_us":8957,"drs_written":1,"lbm_read_time_us":2117,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1538,"lbm_writes_lt_1ms":28,"mutex_wait_us":1047,"num_input_rowsets":2,"peak_mem_usage":25497,"rows_written":171,"spinlock_wait_cycles":14208}
I20260811 02:58:25.274392 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260811 02:58:25.279374 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":165,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:25.279902 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7546 bytes on disk
I20260811 02:58:25.280509 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.302657 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.026s sys 0.005s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":32571,"cfile_cache_miss":31,"cfile_cache_miss_bytes":48813,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":568,"dirs.run_cpu_time_us":1060,"dirs.run_wall_time_us":5319,"drs_written":1,"lbm_read_time_us":1170,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1254,"lbm_writes_lt_1ms":32,"mutex_wait_us":28,"num_input_rowsets":2,"peak_mem_usage":25548,"reinsert_count":0,"rows_written":171,"spinlock_wait_cycles":3840,"update_count":1}
I20260811 02:58:25.303807 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7546 bytes on disk
I20260811 02:58:25.304579 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.314754 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59732:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:25.330143 20165 tablet_service.cc:1467] Processing CreateTablet for tablet b51b265dddf64886baef139cc1a6c058 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.85 [id=3928c872c7ee413a8364bfb515e492f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:25.330642 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b51b265dddf64886baef139cc1a6c058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.337554 20268 tablet_bootstrap.cc:492] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:25.339109 20268 tablet_bootstrap.cc:654] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.341763 20268 tablet_bootstrap.cc:492] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:25.341981 20268 ts_tablet_manager.cc:1397] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260811 02:58:25.342760 20268 raft_consensus.cc:348] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.342995 20268 raft_consensus.cc:374] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.343097 20268 raft_consensus.cc:729] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.343420 20268 consensus_queue.cc:260] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.343645 20268 raft_consensus.cc:388] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.343780 20268 raft_consensus.cc:482] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.343904 20268 raft_consensus.cc:3037] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.346635 20268 raft_consensus.cc:504] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.346928 20268 leader_election.cc:302] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:25.347178 20268 leader_election.cc:288] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.347260 20272 raft_consensus.cc:2781] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.347561 20272 raft_consensus.cc:686] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:25.347780 20268 ts_tablet_manager.cc:1428] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:58:25.347800 20272 consensus_queue.cc:237] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.354465 19970 catalog_manager.cc:5495] T b51b265dddf64886baef139cc1a6c058 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:25.356554 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260811 02:58:25.379014 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.014s sys 0.000s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":48,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1252,"drs_written":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1347,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"spinlock_wait_cycles":3968,"update_count":1}
I20260811 02:58:25.379913 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 508 bytes on disk
I20260811 02:58:25.380506 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.381206 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.251090
I20260811 02:58:25.412284 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.021s sys 0.007s Metrics: {"bytes_written":43351,"cfile_cache_hit":13,"cfile_cache_hit_bytes":32973,"cfile_cache_miss":31,"cfile_cache_miss_bytes":49401,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1986,"dirs.run_cpu_time_us":849,"dirs.run_wall_time_us":4473,"drs_written":1,"lbm_read_time_us":1280,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1345,"lbm_writes_lt_1ms":28,"mutex_wait_us":386,"num_input_rowsets":2,"peak_mem_usage":25974,"rows_written":173}
I20260811 02:58:25.413689 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7651 bytes on disk
I20260811 02:58:25.414320 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.415159 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56262:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:25.427910 20165 tablet_service.cc:1467] Processing CreateTablet for tablet cff07700155c4266aaca0cbccac7c166 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.86 [id=259756ead9d94acb88c7a4946e887b51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:25.428439 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cff07700155c4266aaca0cbccac7c166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.433475 20268 tablet_bootstrap.cc:492] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:25.434955 20268 tablet_bootstrap.cc:654] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.437137 20268 tablet_bootstrap.cc:492] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:25.437319 20268 ts_tablet_manager.cc:1397] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:58:25.438107 20268 raft_consensus.cc:348] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.438308 20268 raft_consensus.cc:374] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.438413 20268 raft_consensus.cc:729] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.438668 20268 consensus_queue.cc:260] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.438874 20268 raft_consensus.cc:388] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.438980 20268 raft_consensus.cc:482] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.439105 20268 raft_consensus.cc:3037] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.441274 20268 raft_consensus.cc:504] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.441577 20268 leader_election.cc:302] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:25.441836 20268 leader_election.cc:288] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.441917 20272 raft_consensus.cc:2781] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.442054 20272 raft_consensus.cc:686] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:25.442297 20272 consensus_queue.cc:237] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.442428 20268 ts_tablet_manager.cc:1428] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:25.447971 19970 catalog_manager.cc:5495] T cff07700155c4266aaca0cbccac7c166 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:25.489490 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56276:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:25.502473 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 26b9007da3284b4fa2fc1b9fe45c8022 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.87 [id=c72872800d8e471ab05d094ff6e53215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:25.502930 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b9007da3284b4fa2fc1b9fe45c8022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.507545 20268 tablet_bootstrap.cc:492] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:25.509126 20268 tablet_bootstrap.cc:654] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.511463 20268 tablet_bootstrap.cc:492] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:25.511672 20268 ts_tablet_manager.cc:1397] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260811 02:58:25.512312 20268 raft_consensus.cc:348] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.512526 20268 raft_consensus.cc:374] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.512629 20268 raft_consensus.cc:729] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.512887 20268 consensus_queue.cc:260] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.513094 20268 raft_consensus.cc:388] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.513216 20268 raft_consensus.cc:482] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.513339 20268 raft_consensus.cc:3037] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.515380 20268 raft_consensus.cc:504] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.515676 20268 leader_election.cc:302] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:25.515893 20268 leader_election.cc:288] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.515985 20272 raft_consensus.cc:2781] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.516258 20272 raft_consensus.cc:686] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:25.516469 20268 ts_tablet_manager.cc:1428] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:25.516570 20272 consensus_queue.cc:237] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.523401 19970 catalog_manager.cc:5495] T 26b9007da3284b4fa2fc1b9fe45c8022 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:25.531540 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:25.543654 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":64,"dirs.run_cpu_time_us":222,"dirs.run_wall_time_us":1329,"drs_written":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1722,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:25.544582 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242109
I20260811 02:58:25.555627 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:25.575827 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.015s sys 0.011s Metrics: {"bytes_written":44226,"cfile_cache_hit":13,"cfile_cache_hit_bytes":33713,"cfile_cache_miss":31,"cfile_cache_miss_bytes":50473,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":637,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":1327,"drs_written":1,"lbm_read_time_us":1053,"lbm_reads_lt_1ms":67,"lbm_write_time_us":916,"lbm_writes_lt_1ms":28,"mutex_wait_us":101,"num_input_rowsets":2,"peak_mem_usage":26487,"rows_written":177,"spinlock_wait_cycles":896}
I20260811 02:58:25.576637 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7810 bytes on disk
I20260811 02:58:25.577237 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.579069 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.005s sys 0.011s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":82,"dirs.run_cpu_time_us":268,"dirs.run_wall_time_us":1257,"drs_written":1,"lbm_read_time_us":144,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1504,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:25.580372 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 517 bytes on disk
I20260811 02:58:25.581138 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.581931 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242109
I20260811 02:58:25.584676 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56280:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:25.600685 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 9f0a395a18914354b87c24b6a71ebc7d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.88 [id=4278060a7f8b4daea7f7a3477ce20cc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:25.601208 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f0a395a18914354b87c24b6a71ebc7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.606792 20268 tablet_bootstrap.cc:492] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:25.608194 20268 tablet_bootstrap.cc:654] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.610792 20268 tablet_bootstrap.cc:492] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:25.610978 20268 ts_tablet_manager.cc:1397] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:25.611949 20268 raft_consensus.cc:348] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.612190 20268 raft_consensus.cc:374] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.612291 20268 raft_consensus.cc:729] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.612552 20268 consensus_queue.cc:260] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.612855 20268 raft_consensus.cc:388] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.612950 20268 raft_consensus.cc:482] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.613086 20268 raft_consensus.cc:3037] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.616245 20268 raft_consensus.cc:504] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.616577 20268 leader_election.cc:302] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:25.616827 20268 leader_election.cc:288] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.616930 20272 raft_consensus.cc:2781] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.617110 20272 raft_consensus.cc:686] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:25.617385 20272 consensus_queue.cc:237] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.617502 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.033s sys 0.001s Metrics: {"bytes_written":44226,"cfile_cache_hit":13,"cfile_cache_hit_bytes":33713,"cfile_cache_miss":31,"cfile_cache_miss_bytes":50473,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1006,"dirs.run_cpu_time_us":232,"dirs.run_wall_time_us":1565,"drs_written":1,"lbm_read_time_us":2054,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1118,"lbm_writes_lt_1ms":28,"mutex_wait_us":32,"num_input_rowsets":2,"peak_mem_usage":26487,"rows_written":177}
I20260811 02:58:25.618845 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7810 bytes on disk
I20260811 02:58:25.619626 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.620286 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260811 02:58:25.626302 20268 ts_tablet_manager.cc:1428] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
I20260811 02:58:25.628646 19970 catalog_manager.cc:5495] T 9f0a395a18914354b87c24b6a71ebc7d P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:25.634610 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.010s sys 0.003s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":1356,"dirs.run_cpu_time_us":204,"dirs.run_wall_time_us":1858,"drs_written":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1038,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:25.635761 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.196945
I20260811 02:58:25.665351 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:25.670415 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.028s sys 0.005s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":34058,"cfile_cache_miss":28,"cfile_cache_miss_bytes":50392,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":584,"dirs.run_cpu_time_us":1135,"dirs.run_wall_time_us":6200,"drs_written":1,"lbm_read_time_us":1384,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1628,"lbm_writes_lt_1ms":32,"mutex_wait_us":26,"num_input_rowsets":2,"peak_mem_usage":26521,"reinsert_count":0,"rows_written":179,"spinlock_wait_cycles":13312,"update_count":1}
I20260811 02:58:25.671860 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7878 bytes on disk
I20260811 02:58:25.672643 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.680622 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.014s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":64,"dirs.run_cpu_time_us":173,"dirs.run_wall_time_us":1424,"drs_written":1,"lbm_read_time_us":109,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1259,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:25.681980 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 522 bytes on disk
I20260811 02:58:25.682742 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":123,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.683770 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242032
I20260811 02:58:25.688328 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56294:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:25.706269 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 72dfa3c1aa084f2ca550349b7c7ce32d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.89 [id=adf521106dd34bf09374b028b54bcf4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:25.706805 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72dfa3c1aa084f2ca550349b7c7ce32d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.713349 20268 tablet_bootstrap.cc:492] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:25.715288 20268 tablet_bootstrap.cc:654] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.717967 20268 tablet_bootstrap.cc:492] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:25.718187 20268 ts_tablet_manager.cc:1397] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:25.719058 20268 raft_consensus.cc:348] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.719340 20268 raft_consensus.cc:374] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.719450 20268 raft_consensus.cc:729] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.719774 20268 consensus_queue.cc:260] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.719983 20268 raft_consensus.cc:388] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.720094 20268 raft_consensus.cc:482] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.720216 20268 raft_consensus.cc:3037] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.723126 20268 raft_consensus.cc:504] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.723426 20268 leader_election.cc:302] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:25.723695 20268 leader_election.cc:288] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.723774 20272 raft_consensus.cc:2781] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.723995 20272 raft_consensus.cc:686] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:25.724238 20272 consensus_queue.cc:237] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.725250 20268 ts_tablet_manager.cc:1428] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260811 02:58:25.730008 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.046s user 0.031s sys 0.005s Metrics: {"bytes_written":44649,"cfile_cache_hit":12,"cfile_cache_hit_bytes":34091,"cfile_cache_miss":28,"cfile_cache_miss_bytes":50669,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1844,"dirs.run_cpu_time_us":553,"dirs.run_wall_time_us":3347,"drs_written":1,"lbm_read_time_us":1366,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1214,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":26690,"rows_written":179}
I20260811 02:58:25.731487 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7890 bytes on disk
I20260811 02:58:25.734354 19970 catalog_manager.cc:5495] T 72dfa3c1aa084f2ca550349b7c7ce32d P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:25.736683 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":157,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.737649 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000016
I20260811 02:58:25.759510 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.013s sys 0.002s Metrics: {"bytes_written":335,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":61,"dirs.run_cpu_time_us":243,"dirs.run_wall_time_us":1396,"drs_written":1,"lbm_read_time_us":131,"lbm_reads_lt_1ms":4,"lbm_write_time_us":6768,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:25.760725 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.463347
I20260811 02:58:25.771579 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.006s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":579,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":90,"lbm_read_time_us":521,"lbm_reads_lt_1ms":25,"lbm_write_time_us":330,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260811 02:58:25.772501 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:25.773262 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.774329 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.168412
I20260811 02:58:25.807529 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56296:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:25.826722 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.052s user 0.036s sys 0.004s Metrics: {"bytes_written":45097,"cfile_cache_hit":13,"cfile_cache_hit_bytes":34494,"cfile_cache_miss":31,"cfile_cache_miss_bytes":51368,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1767,"dirs.run_cpu_time_us":696,"dirs.run_wall_time_us":3462,"drs_written":1,"lbm_read_time_us":1481,"lbm_reads_lt_1ms":63,"lbm_write_time_us":2088,"lbm_writes_lt_1ms":28,"mutex_wait_us":25,"num_input_rowsets":2,"peak_mem_usage":26964,"rows_written":181}
I20260811 02:58:25.827505 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 49ec5d51ae8d4648ac02bd269d37a52d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.90 [id=b9b7b356689c43e49afd3bda8415c433]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:25.828115 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ec5d51ae8d4648ac02bd269d37a52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.828203 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:25.830227 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7969 bytes on disk
I20260811 02:58:25.834424 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":109,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.835580 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000021
I20260811 02:58:25.837990 20268 tablet_bootstrap.cc:492] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:25.840081 20268 tablet_bootstrap.cc:654] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.843417 20268 tablet_bootstrap.cc:492] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:25.843601 20268 ts_tablet_manager.cc:1397] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260811 02:58:25.843364 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.009s sys 0.004s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":991,"dirs.run_cpu_time_us":215,"dirs.run_wall_time_us":1323,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1175,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:25.844214 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.213922
I20260811 02:58:25.844290 20268 raft_consensus.cc:348] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.844551 20268 raft_consensus.cc:374] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.844645 20268 raft_consensus.cc:729] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.844893 20268 consensus_queue.cc:260] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.845104 20268 raft_consensus.cc:388] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.845193 20268 raft_consensus.cc:482] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.845309 20268 raft_consensus.cc:3037] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.847724 20268 raft_consensus.cc:504] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.848017 20268 leader_election.cc:302] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:25.848237 20268 leader_election.cc:288] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.848352 20272 raft_consensus.cc:2781] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.848491 20272 raft_consensus.cc:686] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:25.848747 20272 consensus_queue.cc:237] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.849789 20268 ts_tablet_manager.cc:1428] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:58:25.851521 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.012s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":54,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1114,"drs_written":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4,"lbm_write_time_us":996,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":1408}
I20260811 02:58:25.853046 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:25.853915 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.854961 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.164981
I20260811 02:58:25.866277 19968 catalog_manager.cc:5495] T 49ec5d51ae8d4648ac02bd269d37a52d P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:25.892236 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.034s sys 0.000s Metrics: {"bytes_written":45490,"cfile_cache_hit":12,"cfile_cache_hit_bytes":34794,"cfile_cache_miss":28,"cfile_cache_miss_bytes":51670,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":261,"dirs.run_cpu_time_us":221,"dirs.run_wall_time_us":1321,"drs_written":1,"lbm_read_time_us":1229,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1764,"lbm_writes_lt_1ms":28,"mutex_wait_us":29,"num_input_rowsets":2,"peak_mem_usage":27030,"rows_written":183,"spinlock_wait_cycles":896}
I20260811 02:58:25.894393 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8035 bytes on disk
I20260811 02:58:25.895154 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":158,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.899331 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000004
I20260811 02:58:25.900317 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.026s sys 0.008s Metrics: {"bytes_written":45490,"cfile_cache_hit":12,"cfile_cache_hit_bytes":34798,"cfile_cache_miss":28,"cfile_cache_miss_bytes":51446,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":79,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":1166,"drs_written":1,"lbm_read_time_us":1311,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1357,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":27033,"rows_written":183}
I20260811 02:58:25.902143 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260811 02:58:25.909106 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.003s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":244,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:25.919664 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.002s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":193,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:25.920190 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8035 bytes on disk
I20260811 02:58:25.920755 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.924038 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:25.934718 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.009s sys 0.000s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":74,"dirs.run_cpu_time_us":286,"dirs.run_wall_time_us":1352,"drs_written":1,"lbm_read_time_us":129,"lbm_reads_lt_1ms":4,"lbm_write_time_us":738,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260811 02:58:25.935704 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 440 bytes on disk
I20260811 02:58:25.936434 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":118,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.937815 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.176841
I20260811 02:58:25.945966 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56298:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:25.961925 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 928f58480909480f9028de606a45676b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.91 [id=0cb9d8e85a8c417daada6d840c0f7c9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:25.962482 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 928f58480909480f9028de606a45676b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.968139 20268 tablet_bootstrap.cc:492] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:25.969810 20268 tablet_bootstrap.cc:654] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.969563 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.026s sys 0.004s Metrics: {"bytes_written":45527,"cfile_cache_hit":13,"cfile_cache_hit_bytes":34820,"cfile_cache_miss":31,"cfile_cache_miss_bytes":52090,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":853,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":1632,"drs_written":1,"lbm_read_time_us":1299,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1129,"lbm_writes_lt_1ms":28,"mutex_wait_us":50,"num_input_rowsets":2,"peak_mem_usage":27256,"rows_written":183,"spinlock_wait_cycles":896}
I20260811 02:58:25.970517 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8049 bytes on disk
I20260811 02:58:25.971385 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4}
I20260811 02:58:25.971971 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260811 02:58:25.973021 20268 tablet_bootstrap.cc:492] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:25.973196 20268 ts_tablet_manager.cc:1397] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260811 02:58:25.973963 20268 raft_consensus.cc:348] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.974231 20268 raft_consensus.cc:374] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.974313 20268 raft_consensus.cc:729] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.974597 20268 consensus_queue.cc:260] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.974833 20268 raft_consensus.cc:388] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.974949 20268 raft_consensus.cc:482] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.975051 20268 raft_consensus.cc:3037] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.977059 20268 raft_consensus.cc:504] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.977301 20268 leader_election.cc:302] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:25.977576 20268 leader_election.cc:288] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.977679 20272 raft_consensus.cc:2781] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.977895 20272 raft_consensus.cc:686] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:25.978144 20272 consensus_queue.cc:237] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:25.978646 20268 ts_tablet_manager.cc:1428] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 02:58:25.984385 19970 catalog_manager.cc:5495] T 928f58480909480f9028de606a45676b P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:25.985316 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.007s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":156,"dirs.run_wall_time_us":1482,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":707,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:25.987897 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.198635
I20260811 02:58:26.021849 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.022s sys 0.005s Metrics: {"bytes_written":45973,"cfile_cache_hit":12,"cfile_cache_hit_bytes":35163,"cfile_cache_miss":28,"cfile_cache_miss_bytes":51971,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":960,"dirs.run_cpu_time_us":243,"dirs.run_wall_time_us":1548,"drs_written":1,"lbm_read_time_us":1119,"lbm_reads_lt_1ms":60,"lbm_write_time_us":881,"lbm_writes_lt_1ms":28,"mutex_wait_us":166,"num_input_rowsets":2,"peak_mem_usage":27289,"rows_written":185,"spinlock_wait_cycles":2048}
I20260811 02:58:26.022902 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8133 bytes on disk
I20260811 02:58:26.023572 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.034940 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56308:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:26.049186 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 48a08c7e9e64451d83e929108c73a164 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.92 [id=caa6ec908d89487a9b66df01f859291b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.049794 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48a08c7e9e64451d83e929108c73a164. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.055265 20268 tablet_bootstrap.cc:492] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:26.056823 20268 tablet_bootstrap.cc:654] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.059226 20268 tablet_bootstrap.cc:492] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:26.059384 20268 ts_tablet_manager.cc:1397] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:26.060048 20268 raft_consensus.cc:348] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.060220 20268 raft_consensus.cc:374] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.060336 20268 raft_consensus.cc:729] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.060572 20268 consensus_queue.cc:260] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.060808 20268 raft_consensus.cc:388] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:26.060904 20268 raft_consensus.cc:482] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:26.061069 20268 raft_consensus.cc:3037] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.063272 20268 raft_consensus.cc:504] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.063568 20268 leader_election.cc:302] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:26.063777 20268 leader_election.cc:288] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:26.063968 20272 raft_consensus.cc:2781] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.064162 20272 raft_consensus.cc:686] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:26.064400 20268 ts_tablet_manager.cc:1428] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:26.064435 20272 consensus_queue.cc:237] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.071607 19968 catalog_manager.cc:5495] T 48a08c7e9e64451d83e929108c73a164 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.109333 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56314:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:26.122927 20165 tablet_service.cc:1467] Processing CreateTablet for tablet fb6cb26b1962491a9171d799a4a07e2d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.93 [id=660e4bf280cf46bbb0c0d62dab6059c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.123373 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb6cb26b1962491a9171d799a4a07e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.128333 20268 tablet_bootstrap.cc:492] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:26.130077 20268 tablet_bootstrap.cc:654] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.132320 20268 tablet_bootstrap.cc:492] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:26.132516 20268 ts_tablet_manager.cc:1397] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:26.133342 20268 raft_consensus.cc:348] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.133561 20268 raft_consensus.cc:374] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.133687 20268 raft_consensus.cc:729] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.133924 20268 consensus_queue.cc:260] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.134135 20268 raft_consensus.cc:388] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:26.134241 20268 raft_consensus.cc:482] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:26.134336 20268 raft_consensus.cc:3037] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.136199 20268 raft_consensus.cc:504] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.136548 20268 leader_election.cc:302] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:26.136807 20268 leader_election.cc:288] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:26.136862 20272 raft_consensus.cc:2781] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.137079 20272 raft_consensus.cc:686] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:26.137318 20272 consensus_queue.cc:237] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.137600 20268 ts_tablet_manager.cc:1428] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260811 02:58:26.143666 19970 catalog_manager.cc:5495] T fb6cb26b1962491a9171d799a4a07e2d P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.160512 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000088
I20260811 02:58:26.171797 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000089
I20260811 02:58:26.178562 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.011s sys 0.005s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":73,"dirs.run_cpu_time_us":260,"dirs.run_wall_time_us":1455,"drs_written":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4,"lbm_write_time_us":785,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:26.179461 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 520 bytes on disk
I20260811 02:58:26.181315 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":225,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.184641 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242913
I20260811 02:58:26.188455 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.005s sys 0.008s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":87,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1293,"drs_written":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1281,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:26.189652 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 520 bytes on disk
I20260811 02:58:26.190344 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.191083 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242913
I20260811 02:58:26.200089 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56326:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:26.214802 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.028s sys 0.000s Metrics: {"bytes_written":46848,"cfile_cache_hit":13,"cfile_cache_hit_bytes":35933,"cfile_cache_miss":31,"cfile_cache_miss_bytes":53683,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":257,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":7293,"drs_written":1,"lbm_read_time_us":1631,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1035,"lbm_writes_lt_1ms":28,"mutex_wait_us":59,"num_input_rowsets":2,"peak_mem_usage":28023,"rows_written":189}
I20260811 02:58:26.217056 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8284 bytes on disk
I20260811 02:58:26.220270 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.222273 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260811 02:58:26.226323 20165 tablet_service.cc:1467] Processing CreateTablet for tablet ce5daf39f396483992b0c233a1c53b6a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.94 [id=0a016c93c23e44148c036774357087df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.226842 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce5daf39f396483992b0c233a1c53b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.228293 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.031s sys 0.001s Metrics: {"bytes_written":46848,"cfile_cache_hit":13,"cfile_cache_hit_bytes":35933,"cfile_cache_miss":31,"cfile_cache_miss_bytes":53683,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":700,"dirs.run_cpu_time_us":149,"dirs.run_wall_time_us":1096,"drs_written":1,"lbm_read_time_us":1580,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1481,"lbm_writes_lt_1ms":28,"mutex_wait_us":46,"num_input_rowsets":2,"peak_mem_usage":28023,"rows_written":189,"spinlock_wait_cycles":2816}
I20260811 02:58:26.229429 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260811 02:58:26.235107 20268 tablet_bootstrap.cc:492] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:26.236852 20268 tablet_bootstrap.cc:654] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.239379 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":165,"dirs.run_wall_time_us":985,"drs_written":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1060,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:26.240388 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:26.240788 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.001s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":50,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":3547,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":893,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:26.241042 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":109,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.241569 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8519 bytes on disk
I20260811 02:58:26.241690 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.224197
I20260811 02:58:26.242331 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":172,"lbm_reads_lt_1ms":8}
I20260811 02:58:26.242894 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.224197
I20260811 02:58:26.255175 20268 tablet_bootstrap.cc:492] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:26.255446 20268 ts_tablet_manager.cc:1397] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.000s
I20260811 02:58:26.256354 20268 raft_consensus.cc:348] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.256659 20268 raft_consensus.cc:374] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.256801 20268 raft_consensus.cc:729] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.257117 20268 consensus_queue.cc:260] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.257411 20268 raft_consensus.cc:388] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:26.257562 20268 raft_consensus.cc:482] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:26.257766 20268 raft_consensus.cc:3037] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.260699 20268 raft_consensus.cc:504] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.261090 20268 leader_election.cc:302] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:26.261442 20272 raft_consensus.cc:2781] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.261601 20272 raft_consensus.cc:686] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:26.262833 20272 consensus_queue.cc:237] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.268366 20268 leader_election.cc:288] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:26.269341 20268 ts_tablet_manager.cc:1428] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260811 02:58:26.272935 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.028s sys 0.000s Metrics: {"bytes_written":47242,"cfile_cache_hit":12,"cfile_cache_hit_bytes":36278,"cfile_cache_miss":28,"cfile_cache_miss_bytes":53568,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":845,"dirs.run_cpu_time_us":750,"dirs.run_wall_time_us":4594,"drs_written":1,"lbm_read_time_us":1096,"lbm_reads_lt_1ms":56,"lbm_write_time_us":859,"lbm_writes_lt_1ms":28,"mutex_wait_us":304,"num_input_rowsets":2,"peak_mem_usage":28057,"rows_written":191}
I20260811 02:58:26.274019 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8353 bytes on disk
I20260811 02:58:26.274588 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:26.275154 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.276145 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.033s sys 0.000s Metrics: {"bytes_written":47242,"cfile_cache_hit":12,"cfile_cache_hit_bytes":36278,"cfile_cache_miss":28,"cfile_cache_miss_bytes":53568,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":595,"dirs.run_cpu_time_us":215,"dirs.run_wall_time_us":1026,"drs_written":1,"lbm_read_time_us":1492,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1139,"lbm_writes_lt_1ms":28,"mutex_wait_us":100,"num_input_rowsets":2,"peak_mem_usage":28057,"rows_written":191}
I20260811 02:58:26.277288 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8353 bytes on disk
I20260811 02:58:26.277983 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.284132 19970 catalog_manager.cc:5495] T ce5daf39f396483992b0c233a1c53b6a P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.289520 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.010s sys 0.000s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":51,"dirs.run_cpu_time_us":158,"dirs.run_wall_time_us":1035,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1102,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:26.290571 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242072
I20260811 02:58:26.324021 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.024s sys 0.007s Metrics: {"bytes_written":390,"cfile_cache_hit":12,"cfile_cache_hit_bytes":36274,"cfile_cache_miss":28,"cfile_cache_miss_bytes":53804,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":893,"dirs.run_cpu_time_us":265,"dirs.run_wall_time_us":1340,"drs_written":1,"lbm_read_time_us":1027,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1213,"lbm_writes_lt_1ms":32,"mutex_wait_us":161,"num_input_rowsets":2,"peak_mem_usage":28055,"reinsert_count":0,"rows_written":191,"update_count":2}
I20260811 02:58:26.325090 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8353 bytes on disk
I20260811 02:58:26.325721 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.354122 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56342:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.95"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:26.369226 20165 tablet_service.cc:1467] Processing CreateTablet for tablet f57d1ee11a864ad4863edca5979b8da9 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.95 [id=bdce37a2e7e04c76bc98ccb1f96fe514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.369789 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f57d1ee11a864ad4863edca5979b8da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.374864 20268 tablet_bootstrap.cc:492] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:26.376863 20268 tablet_bootstrap.cc:654] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.380023 20268 tablet_bootstrap.cc:492] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:26.380169 20268 ts_tablet_manager.cc:1397] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 02:58:26.380754 20268 raft_consensus.cc:348] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.380932 20268 raft_consensus.cc:374] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.380995 20268 raft_consensus.cc:729] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.381218 20268 consensus_queue.cc:260] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.381393 20268 raft_consensus.cc:388] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:26.381475 20268 raft_consensus.cc:482] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:26.381542 20268 raft_consensus.cc:3037] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.383880 20268 raft_consensus.cc:504] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.384125 20268 leader_election.cc:302] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:26.384366 20268 leader_election.cc:288] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:26.384418 20272 raft_consensus.cc:2781] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.384601 20272 raft_consensus.cc:686] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:26.384889 20268 ts_tablet_manager.cc:1428] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 02:58:26.384842 20272 consensus_queue.cc:237] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.391340 19970 catalog_manager.cc:5495] T f57d1ee11a864ad4863edca5979b8da9 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.433308 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56350:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.96"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:26.447012 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 33da8634178c495e926615b67c99ee24 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.96 [id=7c8e85d64f7c4f2f8dbe80cc71b0fab1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.447559 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33da8634178c495e926615b67c99ee24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.452273 20268 tablet_bootstrap.cc:492] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:26.454016 20268 tablet_bootstrap.cc:654] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.456192 20268 tablet_bootstrap.cc:492] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:26.456316 20268 ts_tablet_manager.cc:1397] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:26.457086 20268 raft_consensus.cc:348] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.457280 20268 raft_consensus.cc:374] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.457398 20268 raft_consensus.cc:729] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.457664 20268 consensus_queue.cc:260] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.457927 20268 raft_consensus.cc:388] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:26.458036 20268 raft_consensus.cc:482] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:26.458149 20268 raft_consensus.cc:3037] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.460150 20268 raft_consensus.cc:504] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.460448 20268 leader_election.cc:302] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:26.460638 20268 leader_election.cc:288] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:26.460806 20272 raft_consensus.cc:2781] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.461004 20272 raft_consensus.cc:686] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:26.461197 20268 ts_tablet_manager.cc:1428] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 02:58:26.461282 20272 consensus_queue.cc:237] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.468451 19970 catalog_manager.cc:5495] T 33da8634178c495e926615b67c99ee24 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.509459 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56362:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:26.522454 20165 tablet_service.cc:1467] Processing CreateTablet for tablet da39de03621d4b1d8cda2649fbaa8cd4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.97 [id=e9a0d5b2cd284ec49690eede7e99517a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.522964 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da39de03621d4b1d8cda2649fbaa8cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.526032 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:26.527935 20268 tablet_bootstrap.cc:492] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:26.528877 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:26.529706 20268 tablet_bootstrap.cc:654] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.537178 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.000s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":63,"dirs.run_cpu_time_us":236,"dirs.run_wall_time_us":1387,"drs_written":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4,"lbm_write_time_us":904,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:26.538240 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 506 bytes on disk
I20260811 02:58:26.538942 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":116,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.539690 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.236828
I20260811 02:58:26.540149 20268 tablet_bootstrap.cc:492] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:26.540323 20268 ts_tablet_manager.cc:1397] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.009s
I20260811 02:58:26.541265 20268 raft_consensus.cc:348] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.541500 20268 raft_consensus.cc:374] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.541584 20268 raft_consensus.cc:729] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.541883 20268 consensus_queue.cc:260] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.542140 20268 raft_consensus.cc:388] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:26.542253 20268 raft_consensus.cc:482] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:26.542353 20268 raft_consensus.cc:3037] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.542860 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":53,"dirs.run_cpu_time_us":220,"dirs.run_wall_time_us":1110,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1283,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:26.543746 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.236828
I20260811 02:58:26.545150 20268 raft_consensus.cc:504] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.545454 20268 leader_election.cc:302] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:26.545725 20268 leader_election.cc:288] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:26.545796 20272 raft_consensus.cc:2781] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.546028 20272 raft_consensus.cc:686] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:26.546276 20268 ts_tablet_manager.cc:1428] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:58:26.546303 20272 consensus_queue.cc:237] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.553673 19970 catalog_manager.cc:5495] T da39de03621d4b1d8cda2649fbaa8cd4 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.576617 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:26.579444 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.031s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":37342,"cfile_cache_miss":28,"cfile_cache_miss_bytes":55308,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1144,"dirs.run_cpu_time_us":263,"dirs.run_wall_time_us":5108,"drs_written":1,"lbm_read_time_us":1078,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1987,"lbm_writes_lt_1ms":32,"num_input_rowsets":2,"peak_mem_usage":28652,"reinsert_count":0,"rows_written":197,"update_count":1}
I20260811 02:58:26.580312 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8625 bytes on disk
I20260811 02:58:26.581003 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.589059 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":53,"dirs.run_cpu_time_us":185,"dirs.run_wall_time_us":1150,"drs_written":1,"lbm_read_time_us":74,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1043,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:26.589881 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.237262
I20260811 02:58:26.592509 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.028s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":37342,"cfile_cache_miss":28,"cfile_cache_miss_bytes":55308,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":943,"dirs.run_cpu_time_us":153,"dirs.run_wall_time_us":1131,"drs_written":1,"lbm_read_time_us":1018,"lbm_reads_lt_1ms":56,"lbm_write_time_us":2001,"lbm_writes_lt_1ms":32,"mutex_wait_us":213,"num_input_rowsets":2,"peak_mem_usage":28652,"reinsert_count":0,"rows_written":197,"update_count":1}
I20260811 02:58:26.593519 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8625 bytes on disk
I20260811 02:58:26.598202 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":118,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.619454 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56376:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:26.620461 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.027s sys 0.001s Metrics: {"bytes_written":48597,"cfile_cache_hit":13,"cfile_cache_hit_bytes":37413,"cfile_cache_miss":31,"cfile_cache_miss_bytes":55913,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":938,"dirs.run_cpu_time_us":207,"dirs.run_wall_time_us":1991,"drs_written":1,"lbm_read_time_us":1422,"lbm_reads_lt_1ms":67,"lbm_write_time_us":928,"lbm_writes_lt_1ms":28,"mutex_wait_us":58,"num_input_rowsets":2,"peak_mem_usage":29251,"rows_written":197,"spinlock_wait_cycles":13184}
I20260811 02:58:26.621361 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8639 bytes on disk
I20260811 02:58:26.622044 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4}
I20260811 02:58:26.634795 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 756e749ee7034da8b2d7ebd400288810 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.98 [id=c8bb3f1315f34905a92e1d8d33fce807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.635229 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 756e749ee7034da8b2d7ebd400288810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.640040 20268 tablet_bootstrap.cc:492] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:26.641422 20268 tablet_bootstrap.cc:654] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.645323 20268 tablet_bootstrap.cc:492] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:26.645504 20268 ts_tablet_manager.cc:1397] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260811 02:58:26.646214 20268 raft_consensus.cc:348] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.646430 20268 raft_consensus.cc:374] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.646533 20268 raft_consensus.cc:729] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.646823 20268 consensus_queue.cc:260] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.647096 20268 raft_consensus.cc:388] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:26.647202 20268 raft_consensus.cc:482] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:26.647341 20268 raft_consensus.cc:3037] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.649269 20268 raft_consensus.cc:504] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.649542 20268 leader_election.cc:302] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:26.649791 20268 leader_election.cc:288] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:26.649884 20271 raft_consensus.cc:2781] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.650022 20271 raft_consensus.cc:686] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:26.650277 20268 ts_tablet_manager.cc:1428] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:26.650285 20271 consensus_queue.cc:237] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.657478 19970 catalog_manager.cc:5495] T 756e749ee7034da8b2d7ebd400288810 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.698263 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56392:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:26.711371 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 686455c9923e4d6daf8cd8751ac63ceb (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.99 [id=fd58696eeba848448a03dd14770c2db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.711871 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 686455c9923e4d6daf8cd8751ac63ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.717103 20268 tablet_bootstrap.cc:492] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:26.718607 20268 tablet_bootstrap.cc:654] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.721045 20268 tablet_bootstrap.cc:492] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:26.721240 20268 ts_tablet_manager.cc:1397] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:26.721959 20268 raft_consensus.cc:348] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.722179 20268 raft_consensus.cc:374] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.722278 20268 raft_consensus.cc:729] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.722584 20268 consensus_queue.cc:260] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.722779 20268 raft_consensus.cc:388] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:26.722898 20268 raft_consensus.cc:482] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:26.723021 20268 raft_consensus.cc:3037] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.724893 20268 raft_consensus.cc:504] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.725154 20268 leader_election.cc:302] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:26.725370 20268 leader_election.cc:288] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:26.725423 20271 raft_consensus.cc:2781] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.725646 20271 raft_consensus.cc:686] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:26.725900 20271 consensus_queue.cc:237] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.726177 20268 ts_tablet_manager.cc:1428] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:26.732192 19968 catalog_manager.cc:5495] T 686455c9923e4d6daf8cd8751ac63ceb P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.781435 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56394:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.100"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:26.795349 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 26bb722f82504a45a0e815a1f5e33886 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.100 [id=3433a4351d9b463abb8ba6d49b8e38d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.795943 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26bb722f82504a45a0e815a1f5e33886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.801365 20268 tablet_bootstrap.cc:492] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:26.802809 20268 tablet_bootstrap.cc:654] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.804949 20268 tablet_bootstrap.cc:492] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:26.805073 20268 ts_tablet_manager.cc:1397] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:58:26.805775 20268 raft_consensus.cc:348] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.805948 20268 raft_consensus.cc:374] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.806068 20268 raft_consensus.cc:729] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.806321 20268 consensus_queue.cc:260] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.806541 20268 raft_consensus.cc:388] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:26.806628 20268 raft_consensus.cc:482] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:26.806741 20268 raft_consensus.cc:3037] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.808662 20268 raft_consensus.cc:504] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.808964 20268 leader_election.cc:302] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:26.809166 20268 leader_election.cc:288] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:26.809254 20271 raft_consensus.cc:2781] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.809438 20271 raft_consensus.cc:686] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:26.809762 20268 ts_tablet_manager.cc:1428] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:26.809717 20271 consensus_queue.cc:237] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.816735 19970 catalog_manager.cc:5495] T 26bb722f82504a45a0e815a1f5e33886 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.831977 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:26.842947 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":6070,"cfile_init":1,"dirs.queue_time_us":71,"dirs.run_cpu_time_us":232,"dirs.run_wall_time_us":1351,"drs_written":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4,"lbm_write_time_us":720,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:26.843806 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.210477
I20260811 02:58:26.849293 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260811 02:58:26.862277 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6070,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1264,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":995,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:26.863335 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.210477
I20260811 02:58:26.873371 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260811 02:58:26.874388 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56408:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.101"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:26.884662 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.035s sys 0.000s Metrics: {"bytes_written":49902,"cfile_cache_hit":13,"cfile_cache_hit_bytes":38529,"cfile_cache_miss":31,"cfile_cache_miss_bytes":57487,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1157,"dirs.run_cpu_time_us":522,"dirs.run_wall_time_us":2508,"drs_written":1,"lbm_read_time_us":1275,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1160,"lbm_writes_lt_1ms":28,"mutex_wait_us":668,"num_input_rowsets":2,"peak_mem_usage":30020,"rows_written":203}
I20260811 02:58:26.885852 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260811 02:58:26.891448 20165 tablet_service.cc:1467] Processing CreateTablet for tablet a43aeeebbb48406f9b16d02e42fc96ce (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.101 [id=b133c3976e7f47229282cbf30d5f5b3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.891976 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a43aeeebbb48406f9b16d02e42fc96ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.899004 20268 tablet_bootstrap.cc:492] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:26.899708 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.016s sys 0.003s Metrics: {"bytes_written":6070,"cfile_init":1,"dirs.queue_time_us":55,"dirs.run_cpu_time_us":244,"dirs.run_wall_time_us":1920,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":813,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:26.900615 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.210720
I20260811 02:58:26.900806 20268 tablet_bootstrap.cc:654] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.902899 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.013s sys 0.000s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":41,"dirs.run_cpu_time_us":162,"dirs.run_wall_time_us":1165,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1305,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:26.903844 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.540984
I20260811 02:58:26.910279 20268 tablet_bootstrap.cc:492] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:26.910528 20268 ts_tablet_manager.cc:1397] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260811 02:58:26.911229 20268 raft_consensus.cc:348] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.911456 20268 raft_consensus.cc:374] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.911574 20268 raft_consensus.cc:729] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.911891 20268 consensus_queue.cc:260] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.912096 20268 raft_consensus.cc:388] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:26.912233 20268 raft_consensus.cc:482] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:26.912350 20268 raft_consensus.cc:3037] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.914903 20268 raft_consensus.cc:504] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.915246 20268 leader_election.cc:302] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:26.915488 20268 leader_election.cc:288] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:26.915532 20271 raft_consensus.cc:2781] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.916033 20268 ts_tablet_manager.cc:1428] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:26.916208 20271 raft_consensus.cc:686] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:26.916461 20271 consensus_queue.cc:237] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:26.921325 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":461,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":458,"lbm_read_time_us":523,"lbm_reads_lt_1ms":25,"lbm_write_time_us":317,"lbm_writes_lt_1ms":9,"mutex_wait_us":336,"reinsert_count":0,"update_count":1}
I20260811 02:58:26.922075 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9049 bytes on disk
I20260811 02:58:26.922966 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":190,"lbm_reads_lt_1ms":8}
I20260811 02:58:26.923717 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.191016
I20260811 02:58:26.929279 19970 catalog_manager.cc:5495] T a43aeeebbb48406f9b16d02e42fc96ce P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.944355 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.044s user 0.023s sys 0.008s Metrics: {"bytes_written":49902,"cfile_cache_hit":12,"cfile_cache_hit_bytes":38532,"cfile_cache_miss":28,"cfile_cache_miss_bytes":57114,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":359,"dirs.run_cpu_time_us":174,"dirs.run_wall_time_us":1165,"drs_written":1,"lbm_read_time_us":1305,"lbm_reads_lt_1ms":60,"lbm_write_time_us":3892,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"mutex_wait_us":54,"num_input_rowsets":2,"peak_mem_usage":29967,"rows_written":203,"spinlock_wait_cycles":7040}
I20260811 02:58:26.945561 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260811 02:58:26.953969 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.090s user 0.039s sys 0.004s Metrics: {"bytes_written":49902,"cfile_cache_hit":13,"cfile_cache_hit_bytes":38529,"cfile_cache_miss":31,"cfile_cache_miss_bytes":57487,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":12971,"dirs.run_cpu_time_us":1013,"dirs.run_wall_time_us":5838,"drs_written":1,"lbm_read_time_us":1590,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1269,"lbm_writes_lt_1ms":28,"mutex_wait_us":3806,"num_input_rowsets":2,"peak_mem_usage":30020,"rows_written":203}
I20260811 02:58:26.954886 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000026
I20260811 02:58:26.962355 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.011s sys 0.000s Metrics: {"bytes_written":5196,"cfile_init":1,"dirs.queue_time_us":71,"dirs.run_cpu_time_us":170,"dirs.run_wall_time_us":1037,"drs_written":1,"lbm_read_time_us":129,"lbm_reads_lt_1ms":4,"lbm_write_time_us":891,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":1280}
I20260811 02:58:26.963438 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9241 bytes on disk
I20260811 02:58:26.966653 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":200,"lbm_reads_lt_1ms":8}
I20260811 02:58:26.967305 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":5196,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":145,"dirs.run_wall_time_us":1058,"drs_written":1,"lbm_read_time_us":149,"lbm_reads_lt_1ms":4,"lbm_write_time_us":921,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:26.968150 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.191787
I20260811 02:58:26.968473 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.191787
I20260811 02:58:26.975196 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.051s user 0.039s sys 0.006s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":38898,"cfile_cache_miss":31,"cfile_cache_miss_bytes":57434,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1175,"dirs.run_cpu_time_us":1103,"dirs.run_wall_time_us":5816,"drs_written":1,"lbm_read_time_us":1314,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1301,"lbm_writes_lt_1ms":32,"mutex_wait_us":357,"num_input_rowsets":2,"peak_mem_usage":30071,"reinsert_count":0,"rows_written":205,"spinlock_wait_cycles":1280,"thread_start_us":1314,"threads_started":1,"update_count":1}
I20260811 02:58:26.976171 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8956 bytes on disk
I20260811 02:58:26.976871 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.006659 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.033s sys 0.001s Metrics: {"bytes_written":50350,"cfile_cache_hit":12,"cfile_cache_hit_bytes":38903,"cfile_cache_miss":28,"cfile_cache_miss_bytes":57647,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":494,"dirs.run_cpu_time_us":217,"dirs.run_wall_time_us":1508,"drs_written":1,"lbm_read_time_us":1276,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1507,"lbm_writes_lt_1ms":28,"mutex_wait_us":175,"num_input_rowsets":2,"peak_mem_usage":30224,"rows_written":205,"spinlock_wait_cycles":1280}
I20260811 02:58:27.007542 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8966 bytes on disk
I20260811 02:58:27.008224 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":127,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.008325 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56422:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.102"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:27.010720 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.042s user 0.033s sys 0.002s Metrics: {"bytes_written":50350,"cfile_cache_hit":12,"cfile_cache_hit_bytes":38903,"cfile_cache_miss":28,"cfile_cache_miss_bytes":57647,"cfile_init":9,"delta_iterators_relevant":4,"dirs.queue_time_us":649,"dirs.run_cpu_time_us":169,"dirs.run_wall_time_us":1076,"drs_written":1,"lbm_read_time_us":1788,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1204,"lbm_writes_lt_1ms":28,"mutex_wait_us":101,"num_input_rowsets":2,"peak_mem_usage":30224,"rows_written":205,"spinlock_wait_cycles":2304}
I20260811 02:58:27.011989 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8966 bytes on disk
I20260811 02:58:27.013511 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.023869 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 1173c6682f5b4c2c98ed72662e88fb3f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.102 [id=c30e7bdc15ce4d25ae409eeca05d77fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:27.024421 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1173c6682f5b4c2c98ed72662e88fb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:27.029572 20268 tablet_bootstrap.cc:492] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:27.031323 20268 tablet_bootstrap.cc:654] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:27.033499 20268 tablet_bootstrap.cc:492] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:27.033761 20268 ts_tablet_manager.cc:1397] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:58:27.034446 20268 raft_consensus.cc:348] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.034641 20268 raft_consensus.cc:374] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:27.034785 20268 raft_consensus.cc:729] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:27.035128 20268 consensus_queue.cc:260] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.035360 20268 raft_consensus.cc:388] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:27.035470 20268 raft_consensus.cc:482] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:27.035605 20268 raft_consensus.cc:3037] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:27.037568 20268 raft_consensus.cc:504] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.037902 20268 leader_election.cc:302] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:27.038107 20268 leader_election.cc:288] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:27.038197 20271 raft_consensus.cc:2781] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:27.038339 20271 raft_consensus.cc:686] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:27.038681 20268 ts_tablet_manager.cc:1428] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:27.038605 20271 consensus_queue.cc:237] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.045447 19968 catalog_manager.cc:5495] T 1173c6682f5b4c2c98ed72662e88fb3f P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:27.089682 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56430:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.103"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:27.102667 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 2d4cf1fd83f44d03b85c7ffe4b899b6f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.103 [id=917572d80819457194ace91e655c6067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:27.103165 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d4cf1fd83f44d03b85c7ffe4b899b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:27.108090 20268 tablet_bootstrap.cc:492] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:27.109694 20268 tablet_bootstrap.cc:654] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:27.112058 20268 tablet_bootstrap.cc:492] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:27.112249 20268 ts_tablet_manager.cc:1397] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:27.113023 20268 raft_consensus.cc:348] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.113211 20268 raft_consensus.cc:374] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:27.113262 20268 raft_consensus.cc:729] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:27.113519 20268 consensus_queue.cc:260] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.113704 20268 raft_consensus.cc:388] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:27.113772 20268 raft_consensus.cc:482] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:27.113849 20268 raft_consensus.cc:3037] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:27.115729 20268 raft_consensus.cc:504] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.116027 20268 leader_election.cc:302] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:27.116279 20268 leader_election.cc:288] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:27.116384 20271 raft_consensus.cc:2781] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:27.116647 20271 raft_consensus.cc:686] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:27.116803 20268 ts_tablet_manager.cc:1428] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260811 02:58:27.116891 20271 consensus_queue.cc:237] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.122735 19968 catalog_manager.cc:5495] T 2d4cf1fd83f44d03b85c7ffe4b899b6f P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:27.165463 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56432:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.104"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:27.180333 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 93016333bb71463a9c471f88b076088e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.104 [id=a814b032b3ca43119117f9bdc1123382]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:27.180924 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93016333bb71463a9c471f88b076088e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:27.188714 20268 tablet_bootstrap.cc:492] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:27.190512 20268 tablet_bootstrap.cc:654] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:27.193029 20268 tablet_bootstrap.cc:492] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:27.193168 20268 ts_tablet_manager.cc:1397] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:27.193918 20268 raft_consensus.cc:348] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.194159 20268 raft_consensus.cc:374] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:27.194341 20268 raft_consensus.cc:729] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:27.194587 20268 consensus_queue.cc:260] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.194835 20268 raft_consensus.cc:388] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:27.194928 20268 raft_consensus.cc:482] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:27.195048 20268 raft_consensus.cc:3037] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:27.197142 20268 raft_consensus.cc:504] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.197516 20268 leader_election.cc:302] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:27.197788 20268 leader_election.cc:288] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:27.197870 20271 raft_consensus.cc:2781] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:27.198148 20271 raft_consensus.cc:686] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:27.198333 20268 ts_tablet_manager.cc:1428] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:27.198457 20271 consensus_queue.cc:237] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.205359 19968 catalog_manager.cc:5495] T 93016333bb71463a9c471f88b076088e P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:27.227933 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000090
I20260811 02:58:27.240571 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":6072,"cfile_init":1,"dirs.queue_time_us":56,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":1256,"drs_written":1,"lbm_read_time_us":172,"lbm_reads_lt_1ms":4,"lbm_write_time_us":948,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:27.241676 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.206942
I20260811 02:58:27.259256 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260811 02:58:27.264756 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260811 02:58:27.266690 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56434:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.105"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:27.281603 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 1fcc89076f8547b485611c7ad746d74f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.105 [id=7c3510c4377b44809973f4b4eae1583b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:27.282836 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fcc89076f8547b485611c7ad746d74f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:27.284192 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.009s sys 0.000s Metrics: {"bytes_written":6072,"cfile_init":1,"dirs.queue_time_us":782,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":2561,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":826,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:27.284993 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.207168
I20260811 02:58:27.285400 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.010s sys 0.001s Metrics: {"bytes_written":6072,"cfile_init":1,"dirs.queue_time_us":47,"dirs.run_cpu_time_us":164,"dirs.run_wall_time_us":1012,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":855,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:27.286309 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 521 bytes on disk
I20260811 02:58:27.289155 20268 tablet_bootstrap.cc:492] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:27.290086 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.026s sys 0.008s Metrics: {"bytes_written":51652,"cfile_cache_hit":13,"cfile_cache_hit_bytes":40013,"cfile_cache_miss":31,"cfile_cache_miss_bytes":59635,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":617,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1392,"drs_written":1,"lbm_read_time_us":1516,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1561,"lbm_writes_lt_1ms":28,"mutex_wait_us":61,"num_input_rowsets":2,"peak_mem_usage":31044,"rows_written":211,"spinlock_wait_cycles":5376}
I20260811 02:58:27.290916 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000014
I20260811 02:58:27.291087 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":122,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.291237 20268 tablet_bootstrap.cc:654] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:27.293099 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.207168
I20260811 02:58:27.299348 20268 tablet_bootstrap.cc:492] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:27.299554 20268 ts_tablet_manager.cc:1397] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260811 02:58:27.300355 20268 raft_consensus.cc:348] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.300593 20268 raft_consensus.cc:374] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:27.300680 20268 raft_consensus.cc:729] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:27.300940 20268 consensus_queue.cc:260] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.301172 20268 raft_consensus.cc:388] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:27.301267 20268 raft_consensus.cc:482] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:27.301357 20268 raft_consensus.cc:3037] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:27.302801 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.010s sys 0.000s Metrics: {"bytes_written":5159,"cfile_init":1,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":189,"dirs.run_wall_time_us":1153,"drs_written":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4,"lbm_write_time_us":870,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:27.303614 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.187367
I20260811 02:58:27.305233 20268 raft_consensus.cc:504] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.305560 20268 leader_election.cc:302] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:27.305985 20271 raft_consensus.cc:2781] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:27.306176 20271 raft_consensus.cc:686] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:27.306432 20271 consensus_queue.cc:237] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.325807 20268 leader_election.cc:288] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:27.326460 20268 ts_tablet_manager.cc:1428] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260811 02:58:27.342128 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.038s sys 0.000s Metrics: {"bytes_written":52062,"cfile_cache_hit":12,"cfile_cache_hit_bytes":40349,"cfile_cache_miss":28,"cfile_cache_miss_bytes":59485,"cfile_init":9,"delta_iterators_relevant":4,"dirs.queue_time_us":554,"dirs.run_cpu_time_us":193,"dirs.run_wall_time_us":1123,"drs_written":1,"lbm_read_time_us":1415,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1585,"lbm_writes_lt_1ms":28,"mutex_wait_us":29,"num_input_rowsets":2,"peak_mem_usage":31078,"rows_written":213}
I20260811 02:58:27.344313 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9268 bytes on disk
I20260811 02:58:27.344960 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.347965 19968 catalog_manager.cc:5495] T 1fcc89076f8547b485611c7ad746d74f P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:27.358397 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.073s user 0.032s sys 0.000s Metrics: {"bytes_written":51652,"cfile_cache_hit":12,"cfile_cache_hit_bytes":40016,"cfile_cache_miss":28,"cfile_cache_miss_bytes":59254,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":15929,"dirs.run_cpu_time_us":187,"dirs.run_wall_time_us":1328,"drs_written":1,"lbm_read_time_us":1343,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1381,"lbm_writes_lt_1ms":28,"mutex_wait_us":202,"num_input_rowsets":2,"peak_mem_usage":30991,"rows_written":211,"spinlock_wait_cycles":1408}
I20260811 02:58:27.359442 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9202 bytes on disk
I20260811 02:58:27.360126 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.360707 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000021
I20260811 02:58:27.375525 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.082s user 0.035s sys 0.000s Metrics: {"bytes_written":51652,"cfile_cache_hit":12,"cfile_cache_hit_bytes":40016,"cfile_cache_miss":28,"cfile_cache_miss_bytes":59254,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":456,"dirs.run_cpu_time_us":151,"dirs.run_wall_time_us":1018,"drs_written":1,"lbm_read_time_us":1315,"lbm_reads_lt_1ms":56,"lbm_write_time_us":2858,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":30991,"rows_written":211}
I20260811 02:58:27.376655 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9202 bytes on disk
I20260811 02:58:27.376601 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.008s sys 0.002s Metrics: {"bytes_written":335,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":43,"dirs.run_cpu_time_us":278,"dirs.run_wall_time_us":1265,"drs_written":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1093,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260811 02:58:27.377239 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.377483 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 359 bytes on disk
I20260811 02:58:27.377799 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000026
I20260811 02:58:27.378134 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":109,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.378724 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.441953
I20260811 02:58:27.390148 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.007s sys 0.004s Metrics: {"bytes_written":5196,"cfile_init":1,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":1335,"drs_written":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4,"lbm_write_time_us":987,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":896}
I20260811 02:58:27.391114 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.188112
I20260811 02:58:27.391417 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.005s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":614,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":152,"lbm_read_time_us":441,"lbm_reads_lt_1ms":21,"lbm_write_time_us":355,"lbm_writes_lt_1ms":9,"mutex_wait_us":29,"reinsert_count":0,"spinlock_wait_cycles":896,"update_count":1}
I20260811 02:58:27.392028 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.194743
I20260811 02:58:27.427232 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.031s sys 0.004s Metrics: {"bytes_written":52099,"cfile_cache_hit":12,"cfile_cache_hit_bytes":40387,"cfile_cache_miss":28,"cfile_cache_miss_bytes":59817,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":812,"dirs.run_cpu_time_us":205,"dirs.run_wall_time_us":1261,"drs_written":1,"lbm_read_time_us":1539,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1371,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":31459,"rows_written":213}
I20260811 02:58:27.428705 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9304 bytes on disk
I20260811 02:58:27.429348 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.441073 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.036s sys 0.003s Metrics: {"bytes_written":52099,"cfile_cache_hit":13,"cfile_cache_hit_bytes":40421,"cfile_cache_miss":31,"cfile_cache_miss_bytes":60195,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":2062,"dirs.run_cpu_time_us":1170,"dirs.run_wall_time_us":7236,"drs_written":1,"lbm_read_time_us":1302,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1145,"lbm_writes_lt_1ms":28,"mutex_wait_us":287,"num_input_rowsets":2,"peak_mem_usage":31469,"rows_written":213,"spinlock_wait_cycles":6528,"thread_start_us":6342,"threads_started":1}
I20260811 02:58:27.443828 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9304 bytes on disk
I20260811 02:58:27.444602 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":116,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.450848 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56442:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.106"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:27.464184 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 45e0a2c59e694c3a8ef1d898e232e64c (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.106 [id=2e62d3bf481e4bc0b27ed616daab4de4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:27.464644 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e0a2c59e694c3a8ef1d898e232e64c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:27.469440 20268 tablet_bootstrap.cc:492] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:27.470891 20268 tablet_bootstrap.cc:654] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:27.473286 20268 tablet_bootstrap.cc:492] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:27.473412 20268 ts_tablet_manager.cc:1397] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:27.474167 20268 raft_consensus.cc:348] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.474341 20268 raft_consensus.cc:374] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:27.474416 20268 raft_consensus.cc:729] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:27.474618 20268 consensus_queue.cc:260] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.474769 20268 raft_consensus.cc:388] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:27.474826 20268 raft_consensus.cc:482] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:27.474902 20268 raft_consensus.cc:3037] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:27.476722 20268 raft_consensus.cc:504] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.476943 20268 leader_election.cc:302] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:27.477088 20268 leader_election.cc:288] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:27.477233 20271 raft_consensus.cc:2781] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:27.477417 20271 raft_consensus.cc:686] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:27.477648 20268 ts_tablet_manager.cc:1428] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260811 02:58:27.477710 20271 consensus_queue.cc:237] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.496184 19968 catalog_manager.cc:5495] T 45e0a2c59e694c3a8ef1d898e232e64c P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:27.543732 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56450:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.107"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:27.556174 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 9a202ee9f9f9422eaae2535da553a3f6 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.107 [id=40ca540a51204e6697f6a13738760c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:27.556727 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a202ee9f9f9422eaae2535da553a3f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:27.561585 20268 tablet_bootstrap.cc:492] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:27.563136 20268 tablet_bootstrap.cc:654] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:27.565244 20268 tablet_bootstrap.cc:492] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:27.565392 20268 ts_tablet_manager.cc:1397] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:27.566289 20268 raft_consensus.cc:348] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.566496 20268 raft_consensus.cc:374] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:27.566608 20268 raft_consensus.cc:729] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:27.566917 20268 consensus_queue.cc:260] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.567106 20268 raft_consensus.cc:388] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:27.567234 20268 raft_consensus.cc:482] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:27.567350 20268 raft_consensus.cc:3037] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:27.569343 20268 raft_consensus.cc:504] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.569612 20268 leader_election.cc:302] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:27.569919 20268 leader_election.cc:288] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:27.570065 20271 raft_consensus.cc:2781] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:27.570372 20271 raft_consensus.cc:686] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:27.570433 20268 ts_tablet_manager.cc:1428] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260811 02:58:27.570708 20271 consensus_queue.cc:237] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.577701 19968 catalog_manager.cc:5495] T 9a202ee9f9f9422eaae2535da553a3f6 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:27.596572 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260811 02:58:27.605856 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.009s sys 0.000s Metrics: {"bytes_written":5630,"cfile_init":1,"dirs.queue_time_us":69,"dirs.run_cpu_time_us":239,"dirs.run_wall_time_us":1432,"drs_written":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4,"lbm_write_time_us":671,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260811 02:58:27.606628 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 438 bytes on disk
I20260811 02:58:27.607307 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.607960 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.203141
I20260811 02:58:27.629966 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56466:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.108"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:27.640494 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.019s sys 0.011s Metrics: {"bytes_written":52964,"cfile_cache_hit":12,"cfile_cache_hit_bytes":41089,"cfile_cache_miss":28,"cfile_cache_miss_bytes":60769,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":442,"dirs.run_cpu_time_us":224,"dirs.run_wall_time_us":1326,"drs_written":1,"lbm_read_time_us":1100,"lbm_reads_lt_1ms":56,"lbm_write_time_us":981,"lbm_writes_lt_1ms":28,"mutex_wait_us":38,"num_input_rowsets":2,"peak_mem_usage":31589,"rows_written":217,"spinlock_wait_cycles":4352}
I20260811 02:58:27.641510 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9467 bytes on disk
I20260811 02:58:27.642154 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.642872 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260811 02:58:27.645153 20165 tablet_service.cc:1467] Processing CreateTablet for tablet ccfaee0522994fc9bfaf15b962d86c6c (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.108 [id=91271f2d83704c169adcb1d92ffe329f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:27.645700 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccfaee0522994fc9bfaf15b962d86c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:27.650403 20268 tablet_bootstrap.cc:492] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:27.652215 20268 tablet_bootstrap.cc:654] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:27.655081 20268 tablet_bootstrap.cc:492] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:27.655238 20268 ts_tablet_manager.cc:1397] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260811 02:58:27.656054 20268 raft_consensus.cc:348] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.656286 20268 raft_consensus.cc:374] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:27.656394 20268 raft_consensus.cc:729] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:27.656626 20268 consensus_queue.cc:260] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.656872 20268 raft_consensus.cc:388] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:27.656972 20268 raft_consensus.cc:482] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:27.657099 20268 raft_consensus.cc:3037] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:27.659825 20268 raft_consensus.cc:504] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.660106 20268 leader_election.cc:302] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:27.660296 20268 leader_election.cc:288] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:27.660454 20271 raft_consensus.cc:2781] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:27.660635 20271 raft_consensus.cc:686] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:27.660828 20268 ts_tablet_manager.cc:1428] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 02:58:27.660929 20271 consensus_queue.cc:237] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.662985 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.017s sys 0.000s Metrics: {"bytes_written":5159,"cfile_init":1,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":1598,"drs_written":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4,"lbm_write_time_us":700,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:27.667784 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:27.668547 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":127,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.669564 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.216808
I20260811 02:58:27.671239 19968 catalog_manager.cc:5495] T ccfaee0522994fc9bfaf15b962d86c6c P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:27.680503 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:27.694895 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.011s sys 0.000s Metrics: {"bytes_written":6072,"cfile_init":1,"dirs.queue_time_us":430,"dirs.run_cpu_time_us":246,"dirs.run_wall_time_us":1355,"drs_written":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4,"lbm_write_time_us":785,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:27.695611 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000089
I20260811 02:58:27.695946 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.235845
I20260811 02:58:27.707631 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.031s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":41467,"cfile_cache_miss":28,"cfile_cache_miss_bytes":61135,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":928,"dirs.run_cpu_time_us":199,"dirs.run_wall_time_us":1025,"drs_written":1,"lbm_read_time_us":1057,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1220,"lbm_writes_lt_1ms":32,"mutex_wait_us":256,"num_input_rowsets":2,"peak_mem_usage":32056,"reinsert_count":0,"rows_written":219,"update_count":1}
I20260811 02:58:27.708599 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9540 bytes on disk
I20260811 02:58:27.708496 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6072,"cfile_init":1,"dirs.queue_time_us":53,"dirs.run_cpu_time_us":195,"dirs.run_wall_time_us":1244,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1005,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:27.709348 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":142,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.709488 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 518 bytes on disk
I20260811 02:58:27.710500 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.711361 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.235845
I20260811 02:58:27.733879 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.028s sys 0.006s Metrics: {"bytes_written":53391,"cfile_cache_hit":12,"cfile_cache_hit_bytes":41500,"cfile_cache_miss":28,"cfile_cache_miss_bytes":61412,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1150,"dirs.run_cpu_time_us":209,"dirs.run_wall_time_us":1152,"drs_written":1,"lbm_read_time_us":1282,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1205,"lbm_writes_lt_1ms":28,"mutex_wait_us":475,"num_input_rowsets":2,"peak_mem_usage":32225,"rows_written":219}
I20260811 02:58:27.735311 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9556 bytes on disk
I20260811 02:58:27.736147 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":114,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.738144 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56478:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.109"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:27.755453 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.044s user 0.032s sys 0.001s Metrics: {"bytes_written":53391,"cfile_cache_hit":12,"cfile_cache_hit_bytes":41500,"cfile_cache_miss":28,"cfile_cache_miss_bytes":61412,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":710,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":1109,"drs_written":1,"lbm_read_time_us":1304,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1252,"lbm_writes_lt_1ms":28,"mutex_wait_us":22,"num_input_rowsets":2,"peak_mem_usage":32225,"rows_written":219,"spinlock_wait_cycles":10112}
I20260811 02:58:27.757093 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260811 02:58:27.759137 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 898908d12c054d11b41442ab82af6c56 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.109 [id=1baf8dade7c845de9e67b40e4bca1604]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:27.759635 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898908d12c054d11b41442ab82af6c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:27.764760 20268 tablet_bootstrap.cc:492] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:27.766461 20268 tablet_bootstrap.cc:654] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:27.769078 20268 tablet_bootstrap.cc:492] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:27.769309 20268 ts_tablet_manager.cc:1397] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:27.770088 20268 raft_consensus.cc:348] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.770272 20268 raft_consensus.cc:374] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:27.770375 20268 raft_consensus.cc:729] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:27.770658 20268 consensus_queue.cc:260] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.770862 20268 raft_consensus.cc:388] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:27.770984 20268 raft_consensus.cc:482] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:27.771100 20268 raft_consensus.cc:3037] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:27.773892 20268 raft_consensus.cc:504] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.774159 20268 leader_election.cc:302] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:27.774385 20268 leader_election.cc:288] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:27.774504 20271 raft_consensus.cc:2781] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:27.775825 20271 raft_consensus.cc:686] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:27.776111 20271 consensus_queue.cc:237] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.776762 20268 ts_tablet_manager.cc:1428] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260811 02:58:27.777835 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.018s sys 0.000s Metrics: {"bytes_written":5159,"cfile_init":1,"dirs.queue_time_us":65,"dirs.run_cpu_time_us":182,"dirs.run_wall_time_us":1483,"drs_written":1,"lbm_read_time_us":130,"lbm_reads_lt_1ms":4,"lbm_write_time_us":799,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":896}
I20260811 02:58:27.780098 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9791 bytes on disk
I20260811 02:58:27.782382 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":162,"lbm_reads_lt_1ms":8}
I20260811 02:58:27.788242 19968 catalog_manager.cc:5495] T 898908d12c054d11b41442ab82af6c56 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:27.789923 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.194792
I20260811 02:58:27.820645 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.027s sys 0.002s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":41833,"cfile_cache_miss":28,"cfile_cache_miss_bytes":61673,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":586,"dirs.run_cpu_time_us":186,"dirs.run_wall_time_us":1287,"drs_written":1,"lbm_read_time_us":1077,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1186,"lbm_writes_lt_1ms":32,"mutex_wait_us":49,"num_input_rowsets":2,"peak_mem_usage":32312,"reinsert_count":0,"rows_written":221,"update_count":1}
I20260811 02:58:27.821703 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9618 bytes on disk
I20260811 02:58:27.822609 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4}
I20260811 02:58:27.835361 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56482:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.110"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:27.848317 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 4bc1152666894d83a94d8065d5a2019d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.110 [id=bca46f9e09c04fdf9a79e5d59e2cfcad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:27.848811 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bc1152666894d83a94d8065d5a2019d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:27.853547 20268 tablet_bootstrap.cc:492] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:27.855132 20268 tablet_bootstrap.cc:654] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:27.857458 20268 tablet_bootstrap.cc:492] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:27.857626 20268 ts_tablet_manager.cc:1397] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 02:58:27.858392 20268 raft_consensus.cc:348] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.858585 20268 raft_consensus.cc:374] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:27.858707 20268 raft_consensus.cc:729] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:27.858951 20268 consensus_queue.cc:260] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.859156 20268 raft_consensus.cc:388] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:27.859243 20268 raft_consensus.cc:482] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:27.859367 20268 raft_consensus.cc:3037] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:27.861191 20268 raft_consensus.cc:504] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.861480 20268 leader_election.cc:302] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:27.861706 20268 leader_election.cc:288] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:27.861860 20271 raft_consensus.cc:2781] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:27.862088 20271 raft_consensus.cc:686] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:27.862247 20268 ts_tablet_manager.cc:1428] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:58:27.862401 20271 consensus_queue.cc:237] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.871899 19968 catalog_manager.cc:5495] T 4bc1152666894d83a94d8065d5a2019d P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:27.917371 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56492:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.111"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:27.931749 20165 tablet_service.cc:1467] Processing CreateTablet for tablet bc79b5797a534b2b89adafd570a5121f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.111 [id=87b96f5147254d8a9b3b68e4d3bb320d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:27.932310 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc79b5797a534b2b89adafd570a5121f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:27.937217 20268 tablet_bootstrap.cc:492] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:27.938719 20268 tablet_bootstrap.cc:654] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:27.941403 20268 tablet_bootstrap.cc:492] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:27.941665 20268 ts_tablet_manager.cc:1397] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 02:58:27.942390 20268 raft_consensus.cc:348] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.942687 20268 raft_consensus.cc:374] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:27.942802 20268 raft_consensus.cc:729] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:27.943095 20268 consensus_queue.cc:260] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.943331 20268 raft_consensus.cc:388] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:27.943440 20268 raft_consensus.cc:482] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:27.943558 20268 raft_consensus.cc:3037] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:27.945498 20268 raft_consensus.cc:504] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.945775 20268 leader_election.cc:302] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:27.945988 20268 leader_election.cc:288] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:27.946039 20271 raft_consensus.cc:2781] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:27.946216 20271 raft_consensus.cc:686] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:27.946545 20268 ts_tablet_manager.cc:1428] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260811 02:58:27.946448 20271 consensus_queue.cc:237] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:27.953886 19970 catalog_manager.cc:5495] T bc79b5797a534b2b89adafd570a5121f P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:27.960278 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260811 02:58:27.974344 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.011s sys 0.001s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1382,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":901,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260811 02:58:27.975134 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.240261
I20260811 02:58:27.987169 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:28.002267 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.013s sys 0.000s Metrics: {"bytes_written":6072,"cfile_init":1,"dirs.queue_time_us":47,"dirs.run_cpu_time_us":143,"dirs.run_wall_time_us":953,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1258,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:28.003326 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 519 bytes on disk
I20260811 02:58:28.004029 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.005585 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.235121
I20260811 02:58:28.009023 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.028s sys 0.000s Metrics: {"bytes_written":54737,"cfile_cache_hit":14,"cfile_cache_hit_bytes":42602,"cfile_cache_miss":34,"cfile_cache_miss_bytes":63758,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":1380,"dirs.run_cpu_time_us":657,"dirs.run_wall_time_us":4415,"drs_written":1,"lbm_read_time_us":1133,"lbm_reads_lt_1ms":74,"lbm_write_time_us":822,"lbm_writes_lt_1ms":29,"mutex_wait_us":233,"num_input_rowsets":2,"peak_mem_usage":33094,"rows_written":225,"spinlock_wait_cycles":12416,"thread_start_us":232,"threads_started":1}
I20260811 02:58:28.010039 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9890 bytes on disk
I20260811 02:58:28.010601 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.011919 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56496:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.112"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:28.027446 20165 tablet_service.cc:1467] Processing CreateTablet for tablet a0b78148985842b18388b7f34dc455d4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.112 [id=e33ef812039d4c179fe6881520f818f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:28.028009 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b78148985842b18388b7f34dc455d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:28.034488 20268 tablet_bootstrap.cc:492] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:28.035974 20268 tablet_bootstrap.cc:654] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:28.039623 20268 tablet_bootstrap.cc:492] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:28.039768 20268 ts_tablet_manager.cc:1397] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:28.039639 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.030s sys 0.000s Metrics: {"bytes_written":54737,"cfile_cache_hit":12,"cfile_cache_hit_bytes":42608,"cfile_cache_miss":28,"cfile_cache_miss_bytes":63012,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":590,"dirs.run_cpu_time_us":234,"dirs.run_wall_time_us":1304,"drs_written":1,"lbm_read_time_us":1093,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1601,"lbm_writes_lt_1ms":29,"mutex_wait_us":72,"num_input_rowsets":2,"peak_mem_usage":32993,"rows_written":225}
I20260811 02:58:28.040539 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9890 bytes on disk
I20260811 02:58:28.040557 20268 raft_consensus.cc:348] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.040755 20268 raft_consensus.cc:374] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:28.040863 20268 raft_consensus.cc:729] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:28.041129 20268 consensus_queue.cc:260] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.041364 20268 raft_consensus.cc:388] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:28.041488 20268 raft_consensus.cc:482] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:28.041450 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":125,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.041643 20268 raft_consensus.cc:3037] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:28.042167 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260811 02:58:28.043689 20268 raft_consensus.cc:504] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.043921 20268 leader_election.cc:302] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:28.044114 20268 leader_election.cc:288] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:28.044204 20271 raft_consensus.cc:2781] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:28.044355 20271 raft_consensus.cc:686] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:28.044942 20271 consensus_queue.cc:237] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.045182 20268 ts_tablet_manager.cc:1428] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:28.052914 19970 catalog_manager.cc:5495] T a0b78148985842b18388b7f34dc455d4 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:28.057984 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.008s sys 0.003s Metrics: {"bytes_written":5159,"cfile_init":1,"dirs.queue_time_us":206,"dirs.run_cpu_time_us":242,"dirs.run_wall_time_us":4930,"drs_written":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1042,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:28.058908 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260811 02:58:28.059764 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.060454 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.187698
I20260811 02:58:28.073652 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260811 02:58:28.084832 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.007s sys 0.000s Metrics: {"bytes_written":6072,"cfile_init":1,"dirs.queue_time_us":74,"dirs.run_cpu_time_us":197,"dirs.run_wall_time_us":1415,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":765,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:28.085804 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.230533
I20260811 02:58:28.102098 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.031s sys 0.003s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":42970,"cfile_cache_miss":30,"cfile_cache_miss_bytes":63478,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":552,"dirs.run_cpu_time_us":227,"dirs.run_wall_time_us":1477,"drs_written":1,"lbm_read_time_us":1210,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1322,"lbm_writes_lt_1ms":33,"mutex_wait_us":72,"num_input_rowsets":2,"peak_mem_usage":32993,"reinsert_count":0,"rows_written":227,"update_count":1}
I20260811 02:58:28.103133 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10035 bytes on disk
I20260811 02:58:28.104364 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.119359 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.028s sys 0.004s Metrics: {"bytes_written":55164,"cfile_cache_hit":13,"cfile_cache_hit_bytes":42981,"cfile_cache_miss":31,"cfile_cache_miss_bytes":63913,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":770,"dirs.run_cpu_time_us":228,"dirs.run_wall_time_us":1216,"drs_written":1,"lbm_read_time_us":1309,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1148,"lbm_writes_lt_1ms":29,"mutex_wait_us":86,"num_input_rowsets":2,"peak_mem_usage":33297,"rows_written":227,"spinlock_wait_cycles":2304}
I20260811 02:58:28.120146 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10079 bytes on disk
I20260811 02:58:28.120769 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.125978 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56508:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.113"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:28.139124 20165 tablet_service.cc:1467] Processing CreateTablet for tablet c8dbc85edb5d4de39f26145810fe4807 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.113 [id=8eba53a43af8445c91d4b3bf93365e14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:28.139626 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8dbc85edb5d4de39f26145810fe4807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:28.169906 20268 tablet_bootstrap.cc:492] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:28.171633 20268 tablet_bootstrap.cc:654] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:28.173883 20268 tablet_bootstrap.cc:492] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:28.174072 20268 ts_tablet_manager.cc:1397] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:58:28.174746 20268 raft_consensus.cc:348] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.174962 20268 raft_consensus.cc:374] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:28.175067 20268 raft_consensus.cc:729] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:28.175333 20268 consensus_queue.cc:260] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.175524 20268 raft_consensus.cc:388] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:28.175644 20268 raft_consensus.cc:482] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:28.175771 20268 raft_consensus.cc:3037] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:28.177735 20268 raft_consensus.cc:504] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.178027 20268 leader_election.cc:302] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:28.178267 20268 leader_election.cc:288] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:28.178355 20271 raft_consensus.cc:2781] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:28.178550 20271 raft_consensus.cc:686] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:28.178792 20271 consensus_queue.cc:237] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.178992 20268 ts_tablet_manager.cc:1428] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:28.187188 19970 catalog_manager.cc:5495] T c8dbc85edb5d4de39f26145810fe4807 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:28.249923 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56524:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.114"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:28.261441 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:28.263401 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 79df862a59cf4d53bfd1a94b0140ec51 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.114 [id=a6dca784fefb4e97b700b032a9f9eb0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:28.263921 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79df862a59cf4d53bfd1a94b0140ec51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:28.268808 20268 tablet_bootstrap.cc:492] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:28.270419 20268 tablet_bootstrap.cc:654] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:28.271517 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.006s sys 0.003s Metrics: {"bytes_written":6035,"cfile_init":1,"dirs.queue_time_us":76,"dirs.run_cpu_time_us":263,"dirs.run_wall_time_us":1427,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":819,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:28.272341 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.229988
I20260811 02:58:28.273104 20268 tablet_bootstrap.cc:492] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:28.273277 20268 ts_tablet_manager.cc:1397] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 02:58:28.274166 20268 raft_consensus.cc:348] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.274412 20268 raft_consensus.cc:374] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:28.274521 20268 raft_consensus.cc:729] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:28.274758 20268 consensus_queue.cc:260] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.274966 20268 raft_consensus.cc:388] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:28.275070 20268 raft_consensus.cc:482] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:28.275195 20268 raft_consensus.cc:3037] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:28.277123 20268 raft_consensus.cc:504] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.277415 20268 leader_election.cc:302] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:28.277678 20268 leader_election.cc:288] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:28.277776 20271 raft_consensus.cc:2781] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:28.277992 20271 raft_consensus.cc:686] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:28.278306 20271 consensus_queue.cc:237] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.289995 20268 ts_tablet_manager.cc:1428] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:28.293593 19970 catalog_manager.cc:5495] T 79df862a59cf4d53bfd1a94b0140ec51 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:28.301179 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.021s sys 0.004s Metrics: {"bytes_written":55995,"cfile_cache_hit":12,"cfile_cache_hit_bytes":43708,"cfile_cache_miss":30,"cfile_cache_miss_bytes":64762,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":246,"dirs.run_wall_time_us":1354,"drs_written":1,"lbm_read_time_us":1027,"lbm_reads_lt_1ms":62,"lbm_write_time_us":1202,"lbm_writes_lt_1ms":29,"num_input_rowsets":2,"peak_mem_usage":33475,"rows_written":231}
I20260811 02:58:28.302145 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10205 bytes on disk
I20260811 02:58:28.302727 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.303462 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260811 02:58:28.311580 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.003s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":195,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:28.349469 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56540:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.115"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:28.355566 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260811 02:58:28.367791 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 80802fadee164f368ef4ec4f1231b097 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.115 [id=cd033ed485d347bcab7e557b4e3a17e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:28.368310 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80802fadee164f368ef4ec4f1231b097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:28.368924 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.004s sys 0.004s Metrics: {"bytes_written":5630,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":247,"dirs.run_wall_time_us":1383,"drs_written":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1051,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260811 02:58:28.369829 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 440 bytes on disk
I20260811 02:58:28.370489 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.371143 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.223676
I20260811 02:58:28.371789 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:28.375313 20268 tablet_bootstrap.cc:492] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:28.377269 20268 tablet_bootstrap.cc:654] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:28.389173 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.010s sys 0.001s Metrics: {"bytes_written":6035,"cfile_init":1,"dirs.queue_time_us":72,"dirs.run_cpu_time_us":188,"dirs.run_wall_time_us":1350,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1000,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:28.390211 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 509 bytes on disk
I20260811 02:58:28.390904 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.391592 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.229772
I20260811 02:58:28.393221 20268 tablet_bootstrap.cc:492] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:28.393450 20268 ts_tablet_manager.cc:1397] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.014s
I20260811 02:58:28.394346 20268 raft_consensus.cc:348] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.394614 20268 raft_consensus.cc:374] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:28.394703 20268 raft_consensus.cc:729] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:28.395000 20268 consensus_queue.cc:260] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.395237 20268 raft_consensus.cc:388] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:28.395372 20268 raft_consensus.cc:482] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:28.395506 20268 raft_consensus.cc:3037] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:28.398720 20268 raft_consensus.cc:504] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.398988 20268 leader_election.cc:302] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:28.399232 20268 leader_election.cc:288] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:28.399281 20271 raft_consensus.cc:2781] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:28.399747 20268 ts_tablet_manager.cc:1428] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20260811 02:58:28.400053 20271 raft_consensus.cc:686] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:28.400300 20271 consensus_queue.cc:237] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.414016 19970 catalog_manager.cc:5495] T 80802fadee164f368ef4ec4f1231b097 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:28.426465 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.033s sys 0.001s Metrics: {"bytes_written":56450,"cfile_cache_hit":12,"cfile_cache_hit_bytes":44074,"cfile_cache_miss":30,"cfile_cache_miss_bytes":65494,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":713,"dirs.run_cpu_time_us":214,"dirs.run_wall_time_us":2335,"drs_written":1,"lbm_read_time_us":1204,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1192,"lbm_writes_lt_1ms":29,"mutex_wait_us":114,"num_input_rowsets":2,"peak_mem_usage":33493,"rows_written":233,"spinlock_wait_cycles":768}
I20260811 02:58:28.427356 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10291 bytes on disk
I20260811 02:58:28.428005 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.428568 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260811 02:58:28.428840 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.057s user 0.040s sys 0.000s Metrics: {"bytes_written":56032,"cfile_cache_hit":13,"cfile_cache_hit_bytes":43734,"cfile_cache_miss":33,"cfile_cache_miss_bytes":65332,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":3673,"dirs.run_cpu_time_us":283,"dirs.run_wall_time_us":2092,"drs_written":1,"lbm_read_time_us":8948,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":64,"lbm_write_time_us":2698,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":28,"mutex_wait_us":192,"num_input_rowsets":2,"peak_mem_usage":33518,"rows_written":231}
I20260811 02:58:28.429880 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000017
I20260811 02:58:28.435247 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.000s sys 0.003s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":225,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260811 02:58:28.446439 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.010s sys 0.000s Metrics: {"bytes_written":5196,"cfile_init":1,"dirs.queue_time_us":78,"dirs.run_cpu_time_us":283,"dirs.run_wall_time_us":1372,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":6353,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:28.447211 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10611 bytes on disk
I20260811 02:58:28.447952 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":118,"lbm_reads_lt_1ms":8}
I20260811 02:58:28.448634 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.163762
I20260811 02:58:28.470269 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56554:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.116"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:28.488322 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecb9194c95c4e08acbb1120d53b11d3 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.116 [id=d1d7ff3718e74900ac01552232b213ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:28.488829 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecb9194c95c4e08acbb1120d53b11d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:28.492132 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.033s sys 0.007s Metrics: {"bytes_written":56487,"cfile_cache_hit":12,"cfile_cache_hit_bytes":44116,"cfile_cache_miss":30,"cfile_cache_miss_bytes":65592,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":430,"dirs.run_cpu_time_us":229,"dirs.run_wall_time_us":1599,"drs_written":1,"lbm_read_time_us":1010,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1017,"lbm_writes_lt_1ms":29,"mutex_wait_us":41,"num_input_rowsets":2,"peak_mem_usage":33762,"rows_written":233,"spinlock_wait_cycles":4992}
I20260811 02:58:28.492967 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260811 02:58:28.495121 20268 tablet_bootstrap.cc:492] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:28.497028 20268 tablet_bootstrap.cc:654] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:28.500497 20268 tablet_bootstrap.cc:492] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:28.500684 20268 ts_tablet_manager.cc:1397] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:58:28.501586 20268 raft_consensus.cc:348] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.501832 20268 raft_consensus.cc:374] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:28.501927 20268 raft_consensus.cc:729] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:28.502157 20268 consensus_queue.cc:260] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.502382 20268 raft_consensus.cc:388] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:28.502476 20268 raft_consensus.cc:482] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:28.502586 20268 raft_consensus.cc:3037] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:28.504813 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.000s Metrics: {"bytes_written":5159,"cfile_init":1,"dirs.queue_time_us":46,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1350,"drs_written":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4,"lbm_write_time_us":948,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:28.505399 20268 raft_consensus.cc:504] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.505748 20268 leader_election.cc:302] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:28.505759 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10575 bytes on disk
I20260811 02:58:28.505990 20268 leader_election.cc:288] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:28.506111 20271 raft_consensus.cc:2781] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:28.506536 20268 ts_tablet_manager.cc:1428] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 02:58:28.506631 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":177,"lbm_reads_lt_1ms":8}
I20260811 02:58:28.507036 20271 raft_consensus.cc:686] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:28.507320 20271 consensus_queue.cc:237] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.507448 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.160712
I20260811 02:58:28.517824 19970 catalog_manager.cc:5495] T 5ecb9194c95c4e08acbb1120d53b11d3 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:28.540800 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.021s sys 0.012s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":44454,"cfile_cache_miss":30,"cfile_cache_miss_bytes":65874,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":960,"dirs.run_cpu_time_us":229,"dirs.run_wall_time_us":1399,"drs_written":1,"lbm_read_time_us":1245,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1388,"lbm_writes_lt_1ms":33,"mutex_wait_us":251,"num_input_rowsets":2,"peak_mem_usage":34018,"reinsert_count":0,"rows_written":235,"update_count":1}
I20260811 02:58:28.541852 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10378 bytes on disk
I20260811 02:58:28.542418 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.562613 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56562:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.117"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:28.562852 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:28.575721 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.000s Metrics: {"bytes_written":5630,"cfile_init":1,"dirs.queue_time_us":59,"dirs.run_cpu_time_us":188,"dirs.run_wall_time_us":1184,"drs_written":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4,"lbm_write_time_us":938,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260811 02:58:28.576661 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 440 bytes on disk
I20260811 02:58:28.577194 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 73d3b4df920e43c18ecbc7e8e59d801b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.117 [id=17cc13db99b14552bf638a9f788c5aaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:28.577747 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d3b4df920e43c18ecbc7e8e59d801b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:28.583298 20268 tablet_bootstrap.cc:492] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:28.583839 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":131,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.584704 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.170481
I20260811 02:58:28.585153 20268 tablet_bootstrap.cc:654] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:28.589922 20268 tablet_bootstrap.cc:492] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:28.590126 20268 ts_tablet_manager.cc:1397] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 02:58:28.590806 20268 raft_consensus.cc:348] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.590996 20268 raft_consensus.cc:374] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:28.591100 20268 raft_consensus.cc:729] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:28.591391 20268 consensus_queue.cc:260] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.591589 20268 raft_consensus.cc:388] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:28.591701 20268 raft_consensus.cc:482] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:28.591804 20268 raft_consensus.cc:3037] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:28.594168 20268 raft_consensus.cc:504] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.594444 20268 leader_election.cc:302] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:28.594734 20268 leader_election.cc:288] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:28.594789 20271 raft_consensus.cc:2781] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:28.594969 20271 raft_consensus.cc:686] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:28.595216 20271 consensus_queue.cc:237] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.595587 20268 ts_tablet_manager.cc:1428] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:58:28.612444 19968 catalog_manager.cc:5495] T 73d3b4df920e43c18ecbc7e8e59d801b P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:28.635021 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.050s user 0.027s sys 0.018s Metrics: {"bytes_written":56914,"cfile_cache_hit":13,"cfile_cache_hit_bytes":44476,"cfile_cache_miss":33,"cfile_cache_miss_bytes":66424,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1540,"dirs.run_cpu_time_us":1184,"dirs.run_wall_time_us":6366,"drs_written":1,"lbm_read_time_us":1316,"lbm_reads_lt_1ms":65,"lbm_write_time_us":1330,"lbm_writes_lt_1ms":29,"mutex_wait_us":212,"num_input_rowsets":2,"peak_mem_usage":33815,"rows_written":235,"spinlock_wait_cycles":6016,"thread_start_us":3112,"threads_started":2}
I20260811 02:58:28.635979 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000017
I20260811 02:58:28.647106 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":5196,"cfile_init":1,"dirs.queue_time_us":145,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1152,"drs_written":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4,"lbm_write_time_us":866,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:28.647847 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10777 bytes on disk
I20260811 02:58:28.648720 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":188,"lbm_reads_lt_1ms":8}
I20260811 02:58:28.649276 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.193994
I20260811 02:58:28.663981 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56566:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.118"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:28.678921 20165 tablet_service.cc:1467] Processing CreateTablet for tablet 81a13bf6ff4c406fa1cc1d68b45814c0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.118 [id=f9df5397058b4dcda331cdd657b27c69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:28.679428 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81a13bf6ff4c406fa1cc1d68b45814c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:28.684093 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.031s sys 0.000s Metrics: {"bytes_written":57362,"cfile_cache_hit":12,"cfile_cache_hit_bytes":44858,"cfile_cache_miss":30,"cfile_cache_miss_bytes":66672,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":658,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":1309,"drs_written":1,"lbm_read_time_us":1200,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1251,"lbm_writes_lt_1ms":29,"mutex_wait_us":71,"num_input_rowsets":2,"peak_mem_usage":34274,"rows_written":237,"spinlock_wait_cycles":9728}
I20260811 02:58:28.685225 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10461 bytes on disk
I20260811 02:58:28.685906 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":120,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.686205 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260811 02:58:28.686455 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260811 02:58:28.687201 20268 tablet_bootstrap.cc:492] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:28.688737 20268 tablet_bootstrap.cc:654] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:28.691459 20268 tablet_bootstrap.cc:492] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:28.691649 20268 ts_tablet_manager.cc:1397] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 02:58:28.692416 20268 raft_consensus.cc:348] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.692655 20268 raft_consensus.cc:374] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:28.692764 20268 raft_consensus.cc:729] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:28.693045 20268 consensus_queue.cc:260] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.693252 20268 raft_consensus.cc:388] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:28.693369 20268 raft_consensus.cc:482] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:28.693501 20268 raft_consensus.cc:3037] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:28.695647 20268 raft_consensus.cc:504] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.695928 20268 leader_election.cc:302] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:28.696139 20268 leader_election.cc:288] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:28.696220 20271 raft_consensus.cc:2781] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:28.696367 20271 raft_consensus.cc:686] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:28.696643 20271 consensus_queue.cc:237] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.696754 20268 ts_tablet_manager.cc:1428] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:58:28.704998 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.000s Metrics: {"bytes_written":5159,"cfile_init":1,"dirs.queue_time_us":2931,"drs_written":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4,"lbm_write_time_us":919,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:28.705902 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.160962
I20260811 02:58:28.705745 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.010s sys 0.002s Metrics: {"bytes_written":6035,"cfile_init":1,"dirs.queue_time_us":76,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1230,"drs_written":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1000,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:28.706733 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.207081
I20260811 02:58:28.712736 19968 catalog_manager.cc:5495] T 81a13bf6ff4c406fa1cc1d68b45814c0 P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:28.744426 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.030s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":45196,"cfile_cache_miss":30,"cfile_cache_miss_bytes":66854,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1156,"dirs.run_cpu_time_us":227,"dirs.run_wall_time_us":5280,"drs_written":1,"lbm_read_time_us":1298,"lbm_reads_lt_1ms":62,"lbm_write_time_us":1408,"lbm_writes_lt_1ms":33,"mutex_wait_us":41,"num_input_rowsets":2,"peak_mem_usage":34530,"reinsert_count":0,"rows_written":239,"spinlock_wait_cycles":7040,"update_count":1}
I20260811 02:58:28.745453 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10581 bytes on disk
I20260811 02:58:28.746052 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.751117 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.044s user 0.032s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":45189,"cfile_cache_miss":33,"cfile_cache_miss_bytes":67459,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":722,"dirs.run_cpu_time_us":267,"dirs.run_wall_time_us":1200,"drs_written":1,"lbm_read_time_us":1080,"lbm_reads_lt_1ms":69,"lbm_write_time_us":1461,"lbm_writes_lt_1ms":33,"mutex_wait_us":201,"num_input_rowsets":2,"peak_mem_usage":34066,"reinsert_count":0,"rows_written":239,"update_count":1}
I20260811 02:58:28.751909 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10581 bytes on disk
I20260811 02:58:28.752461 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.775300 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56580:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.119"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:28.790942 20165 tablet_service.cc:1467] Processing CreateTablet for tablet e69081d2388347c0802934941e7f841a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.119 [id=c5c903424b4b48eda6a2b96313168ae0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:28.791489 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e69081d2388347c0802934941e7f841a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:28.793299 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260811 02:58:28.796948 20268 tablet_bootstrap.cc:492] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:28.798803 20268 tablet_bootstrap.cc:654] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:28.801688 20268 tablet_bootstrap.cc:492] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02: No bootstrap required, opened a new log
I20260811 02:58:28.801867 20268 ts_tablet_manager.cc:1397] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:28.802656 20268 raft_consensus.cc:348] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.802892 20268 raft_consensus.cc:374] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:28.802968 20268 raft_consensus.cc:729] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Initialized, Role: FOLLOWER
I20260811 02:58:28.803225 20268 consensus_queue.cc:260] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.803416 20268 raft_consensus.cc:388] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:28.803510 20268 raft_consensus.cc:482] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:28.803606 20268 raft_consensus.cc:3037] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:28.804726 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.000s Metrics: {"bytes_written":6035,"cfile_init":1,"dirs.queue_time_us":70,"dirs.run_cpu_time_us":248,"dirs.run_wall_time_us":1484,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":893,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260811 02:58:28.805557 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.234347
I20260811 02:58:28.806137 20268 raft_consensus.cc:504] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.806403 20268 leader_election.cc:302] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c3d6a808e479996d6dcca28229f02; no voters:
I20260811 02:58:28.806644 20268 leader_election.cc:288] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:28.806699 20271 raft_consensus.cc:2781] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:28.807173 20268 ts_tablet_manager.cc:1428] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:28.807659 20271 raft_consensus.cc:686] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c3d6a808e479996d6dcca28229f02, State: Running, Role: LEADER
I20260811 02:58:28.807932 20271 consensus_queue.cc:237] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } }
I20260811 02:58:28.815011 19968 catalog_manager.cc:5495] T e69081d2388347c0802934941e7f841a P 1b2c3d6a808e479996d6dcca28229f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2c3d6a808e479996d6dcca28229f02 (127.13.94.129). New cstate: current_term: 1 leader_uuid: "1b2c3d6a808e479996d6dcca28229f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c3d6a808e479996d6dcca28229f02" member_type: VOTER last_known_addr { host: "127.13.94.129" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:28.842409 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.029s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":45555,"cfile_cache_miss":33,"cfile_cache_miss_bytes":68003,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":833,"dirs.run_cpu_time_us":233,"dirs.run_wall_time_us":1266,"drs_written":1,"lbm_read_time_us":1099,"lbm_reads_lt_1ms":69,"lbm_write_time_us":1420,"lbm_writes_lt_1ms":33,"mutex_wait_us":100,"num_input_rowsets":2,"peak_mem_usage":34931,"reinsert_count":0,"rows_written":241,"spinlock_wait_cycles":896,"update_count":1}
I20260811 02:58:28.843287 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10671 bytes on disk
I20260811 02:58:28.843909 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4}
I20260811 02:58:28.866824 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56590:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.120"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:28.879760 20165 tablet_service.cc:1467] Processing CreateTablet for tablet ecc5e39e7b3945e39e89cfd190a0470e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.120 [id=dc49a610d01c4c7881dd1a229dab224a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:28.880340 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecc5e39e7b3945e39e89cfd190a0470e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:28.885514 20268 tablet_bootstrap.cc:492] T ecc5e39e7b3945e39e89cfd190a0470e P 1b2c3d6a808e479996d6dcca28229f02: Bootstrap starting.
I20260811 02:58:28.887095 20268 tablet_bootstrap.cc:654] T ecc5e39e7b3945e39e89cfd190a0470e P 1b2c3d6a808e479996d6dcca28229f02: Neither blocks nor log segments found. Creating new log.
W20260811 02:58:28.888826 20268 log.cc:1242] Error closing log: Illegal state: Log not open. State: 0
E20260811 02:58:28.890084 20268 ts_tablet_manager.cc:1419] T ecc5e39e7b3945e39e89cfd190a0470e P 1b2c3d6a808e479996d6dcca28229f02: Tablet failed to bootstrap: IO error: Failed to open new log: Insufficient disk space to allocate 8388608 bytes under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/ts-0/wal/wals/ecc5e39e7b3945e39e89cfd190a0470e/.kudutmp.newsegment7WjK2o (519651328 bytes available vs 518351011 bytes reserved) (error 28)
I20260811 02:58:28.890218 20268 ts_tablet_manager.cc:1397] T ecc5e39e7b3945e39e89cfd190a0470e P 1b2c3d6a808e479996d6dcca28229f02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260811 02:58:28.890304 20268 tablet_replica.cc:330] T ecc5e39e7b3945e39e89cfd190a0470e P 1b2c3d6a808e479996d6dcca28229f02: stopping tablet replica
I20260811 02:58:28.890408 20268 raft_consensus.cc:2229] T ecc5e39e7b3945e39e89cfd190a0470e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 02:58:28.890501 20268 raft_consensus.cc:2258] T ecc5e39e7b3945e39e89cfd190a0470e P 1b2c3d6a808e479996d6dcca28229f02 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:28.997057 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:29.003399 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260811 02:58:29.008116 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.006s sys 0.002s Metrics: {"bytes_written":5592,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":230,"dirs.run_wall_time_us":1317,"drs_written":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4,"lbm_write_time_us":699,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260811 02:58:29.008994 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.197064
I20260811 02:58:29.021974 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.017s sys 0.000s Metrics: {"bytes_written":5592,"cfile_init":1,"dirs.queue_time_us":107,"dirs.run_cpu_time_us":290,"dirs.run_wall_time_us":1347,"drs_written":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1672,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260811 02:58:29.022730 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 426 bytes on disk
I20260811 02:58:29.023283 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260811 02:58:29.023815 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.197064
I20260811 02:58:29.036325 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.023s sys 0.004s Metrics: {"bytes_written":58631,"cfile_cache_hit":13,"cfile_cache_hit_bytes":45923,"cfile_cache_miss":33,"cfile_cache_miss_bytes":68603,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":765,"dirs.run_cpu_time_us":248,"dirs.run_wall_time_us":1375,"drs_written":1,"lbm_read_time_us":1035,"lbm_reads_lt_1ms":69,"lbm_write_time_us":977,"lbm_writes_lt_1ms":29,"mutex_wait_us":73,"num_input_rowsets":2,"peak_mem_usage":35188,"rows_written":243}
I20260811 02:58:29.037497 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10747 bytes on disk
I20260811 02:58:29.038100 19954 maintenance_manager.cc:643] P 49f332cd6dbe412892f57f5449b737a9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4}
I20260811 02:58:29.048368 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.024s sys 0.000s Metrics: {"bytes_written":58631,"cfile_cache_hit":13,"cfile_cache_hit_bytes":45923,"cfile_cache_miss":33,"cfile_cache_miss_bytes":68603,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1486,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":1099,"drs_written":1,"lbm_read_time_us":993,"lbm_reads_lt_1ms":65,"lbm_write_time_us":949,"lbm_writes_lt_1ms":29,"mutex_wait_us":141,"num_input_rowsets":2,"peak_mem_usage":35188,"rows_written":243,"spinlock_wait_cycles":1792}
I20260811 02:58:29.049273 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10747 bytes on disk
I20260811 02:58:29.049853 19871 maintenance_manager.cc:643] P 2484d71914e34bb3bbef9adbf21210a4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4}
I20260811 02:58:29.094990 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260811 02:58:29.104915 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.007s sys 0.000s Metrics: {"bytes_written":5159,"cfile_init":1,"dirs.queue_time_us":85,"dirs.run_cpu_time_us":258,"dirs.run_wall_time_us":1408,"drs_written":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4,"lbm_write_time_us":906,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260811 02:58:29.105765 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.189831
I20260811 02:58:29.129513 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.022s sys 0.001s Metrics: {"bytes_written":58631,"cfile_cache_hit":13,"cfile_cache_hit_bytes":45935,"cfile_cache_miss":33,"cfile_cache_miss_bytes":68425,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":792,"dirs.run_cpu_time_us":263,"dirs.run_wall_time_us":1409,"drs_written":1,"lbm_read_time_us":926,"lbm_reads_lt_1ms":69,"lbm_write_time_us":901,"lbm_writes_lt_1ms":29,"mutex_wait_us":83,"num_input_rowsets":2,"peak_mem_usage":35189,"rows_written":243,"spinlock_wait_cycles":7296}
I20260811 02:58:29.130365 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10747 bytes on disk
I20260811 02:58:29.130957 19795 maintenance_manager.cc:643] P 91c72b5787ad4f709936a56d6112a641: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/dynamic_multi_master-test.cc:403: Failure
Failed
Bad status: Timed out: Timed out waiting for Table Creation
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/dynamic_multi_master-test.cc:1111: Failure
Expected: StartClusterWithSysCatalogGCed( &master_hps, {"--consensus_rpc_timeout_ms=2000", "--follower_unavailable_considered_failed_sec=4"}) doesn't generate new fatal failures in the current thread.
Actual: it does.
I20260811 02:58:58.811403 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 20087
I20260811 02:58:58.819725 20229 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 02:58:58.915568 20033 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9: Tablet ecc5e39e7b3945e39e89cfd190a0470e (table Table.TestRecoverDeadMasterSysCatalogCopy/1.120 [id=dc49a610d01c4c7881dd1a229dab224a]) was not created within the allowed timeout. Replacing with a new tablet 1a5a89d107ac40d084d8087b63df8b71
I20260811 02:58:58.947573 20033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9: Sending DeleteTablet for 1 replicas of tablet ecc5e39e7b3945e39e89cfd190a0470e
I20260811 02:58:59.090857 19955 maintenance_manager.cc:419] P 49f332cd6dbe412892f57f5449b737a9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.008356
W20260811 02:58:59.091958 19954 dir_manager.cc:174] Insufficient disk space under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/data/data: will retry after 10 seconds: IO error: Insufficient disk space to allocate 0 bytes under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-2/data/data (516280320 bytes available vs 518351011 bytes reserved) (error 28)
W20260811 02:58:59.092522 19954 tablet_replica_mm_ops.cc:214] T 00000000000000000000000000000000 P 49f332cd6dbe412892f57f5449b737a9: failed to flush MRS: IO error: Failed to open DiskRowSet for flush: tablet 00000000000000000000000000000000: unable to open output file for column entry_type INT8 NOT NULL: No directories could be added (1 dirs total, 0 failed, 1 full): IO error: No directories available in 00000000000000000000000000000000's directory group (1 dirs total, 0 failed, 1 full) (error 28) (error 19)
F20260811 02:58:59.092644 19954 tablet_replica_mm_ops.cc:215] Check failed: tablet->HasBeenStopped() Unrecoverable flush failure caused by error: IO error: Failed to open DiskRowSet for flush: tablet 00000000000000000000000000000000: unable to open output file for column entry_type INT8 NOT NULL: No directories could be added (1 dirs total, 0 failed, 1 full): IO error: No directories available in 00000000000000000000000000000000's directory group (1 dirs total, 0 failed, 1 full) (error 28) (error 19)
*** Check failure stack trace: ***
@ 0x7f4a4c6c09ad google::LogMessage::Fail() at ??:0
@ 0x7f4a4c6c388a google::LogMessage::SendToLog() at ??:0
@ 0x7f4a4c6c04de google::LogMessage::Flush() at ??:0
@ 0x7f4a4c6c1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260811 02:58:59.104739 19872 maintenance_manager.cc:419] P 2484d71914e34bb3bbef9adbf21210a4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.008356
@ 0x7f4a5ba95e7d kudu::tablet::FlushMRSOp::Perform() at ??:0
W20260811 02:58:59.106199 19871 dir_manager.cc:174] Insufficient disk space under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/data/data: will retry after 10 seconds: IO error: Insufficient disk space to allocate 0 bytes under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-1/data/data (516280320 bytes available vs 518351011 bytes reserved) (error 28)
W20260811 02:58:59.106873 19871 tablet_replica_mm_ops.cc:214] T 00000000000000000000000000000000 P 2484d71914e34bb3bbef9adbf21210a4: failed to flush MRS: IO error: Failed to open DiskRowSet for flush: tablet 00000000000000000000000000000000: unable to open output file for column entry_type INT8 NOT NULL: No directories could be added (1 dirs total, 0 failed, 1 full): IO error: No directories available in 00000000000000000000000000000000's directory group (1 dirs total, 0 failed, 1 full) (error 28) (error 19)
F20260811 02:58:59.107007 19871 tablet_replica_mm_ops.cc:215] Check failed: tablet->HasBeenStopped() Unrecoverable flush failure caused by error: IO error: Failed to open DiskRowSet for flush: tablet 00000000000000000000000000000000: unable to open output file for column entry_type INT8 NOT NULL: No directories could be added (1 dirs total, 0 failed, 1 full): IO error: No directories available in 00000000000000000000000000000000's directory group (1 dirs total, 0 failed, 1 full) (error 28) (error 19)
*** Check failure stack trace: ***
@ 0x7f4a4f1ae090 kudu::MaintenanceManager::LaunchOp() at ??:0
@ 0x7f7f09bc99ad google::LogMessage::Fail() at ??:0
@ 0x7f7f09bcc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f4a4f1b0e53 kudu::MaintenanceManager::RunSchedulerThread()::$_3::operator()() at ??:0
@ 0x7f7f09bc94de google::LogMessage::Flush() at ??:0
@ 0x7f4a4f1b0ce9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f7f09bcaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4a4f30c345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7f18f9ee7d kudu::tablet::FlushMRSOp::Perform() at ??:0
@ 0x7f4a4f30f91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f7f0c6b7090 kudu::MaintenanceManager::LaunchOp() at ??:0
@ 0x7f4a4f30f799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f7f0c6b9e53 kudu::MaintenanceManager::RunSchedulerThread()::$_3::operator()() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260811 02:58:59.137809 20164 tablet_service.cc:1467] Processing CreateTablet for tablet 1a5a89d107ac40d084d8087b63df8b71 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.120 [id=dc49a610d01c4c7881dd1a229dab224a]), partition=RANGE (key) PARTITION UNBOUNDED
@ 0x7f4a4f2eaeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4a525c46db start_thread at ??:0
I20260811 02:58:59.142201 20165 tablet_service.cc:1514] Processing DeleteTablet for tablet ecc5e39e7b3945e39e89cfd190a0470e with delete_type TABLET_DATA_DELETED (Replaced by 1a5a89d107ac40d084d8087b63df8b71 at 2026-08-11 02:58:58 UTC) from {username='slave'} at 127.0.0.1:57208
I20260811 02:58:59.142997 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 20087
@ 0x7f7f0c6b9ce9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4a4b55a61f clone at ??:0
@ 0x7f7f0c815345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7f0c81891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f7f0c818799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7f0c7f3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7f0facd6db start_thread at ??:0
@ 0x7f7f08a6361f clone at ??:0
I20260811 02:58:59.187187 19796 maintenance_manager.cc:419] P 91c72b5787ad4f709936a56d6112a641: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.008356
W20260811 02:58:59.188582 19795 dir_manager.cc:174] Insufficient disk space under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/data/data: will retry after 10 seconds: IO error: Insufficient disk space to allocate 0 bytes under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0/minicluster-data/master-0/data/data (516280320 bytes available vs 518351011 bytes reserved) (error 28)
W20260811 02:58:59.189193 19795 tablet_replica_mm_ops.cc:214] T 00000000000000000000000000000000 P 91c72b5787ad4f709936a56d6112a641: failed to flush MRS: IO error: Failed to open DiskRowSet for flush: tablet 00000000000000000000000000000000: unable to open output file for column entry_type INT8 NOT NULL: No directories could be added (1 dirs total, 0 failed, 1 full): IO error: No directories available in 00000000000000000000000000000000's directory group (1 dirs total, 0 failed, 1 full) (error 28) (error 19)
F20260811 02:58:59.189307 19795 tablet_replica_mm_ops.cc:215] Check failed: tablet->HasBeenStopped() Unrecoverable flush failure caused by error: IO error: Failed to open DiskRowSet for flush: tablet 00000000000000000000000000000000: unable to open output file for column entry_type INT8 NOT NULL: No directories could be added (1 dirs total, 0 failed, 1 full): IO error: No directories available in 00000000000000000000000000000000's directory group (1 dirs total, 0 failed, 1 full) (error 28) (error 19)
*** Check failure stack trace: ***
@ 0x7f45b04979ad google::LogMessage::Fail() at ??:0
@ 0x7f45b049a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f45b04974de google::LogMessage::Flush() at ??:0
@ 0x7f45b0498a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f45bf86ce7d kudu::tablet::FlushMRSOp::Perform() at ??:0
@ 0x7f45b2f85090 kudu::MaintenanceManager::LaunchOp() at ??:0
@ 0x7f45b2f87e53 kudu::MaintenanceManager::RunSchedulerThread()::$_3::operator()() at ??:0
W20260811 02:58:59.220894 19794 connection.cc:535] server connection from 127.13.94.129:45291 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 02:58:59.222642 13690 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu pid 19776
@ 0x7f45b2f87ce9 std::_Function_handler<>::_M_invoke() at ??:0
I20260811 02:58:59.224489 19841 generic_service.cc:196] Checking for leaks (request via RPC)
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f45b30e3345 kudu::ThreadPool::DispatchThread() at ??:0
I20260811 02:58:59.534463 13690 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu with pid 19776
2026-08-11T02:58:59Z chronyd exiting
I20260811 02:58:59.590161 13690 test_util.cc:175] -----------------------------------------------
I20260811 02:58:59.590281 13690 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786417032720145-13690-0
[ FAILED ] ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy/1, where GetParam() = 3 (44878 ms)
[----------] 1 test from ParameterizedRecoverMasterTest (44878 ms total)
[----------] 1 test from ParameterizedAutoAddMasterTest
[ RUN ] ParameterizedAutoAddMasterTest.TestBasicAddition/2_orig_masters_secure
Loading random data
Initializing database '/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:58:59 dist-test-slave-p3t5 krb5kdc[21591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:58:59 dist-test-slave-p3t5 krb5kdc[21591](info): set up 2 sockets
Aug 11 02:58:59 dist-test-slave-p3t5 krb5kdc[21591](info): commencing operation
krb5kdc: starting...
W20260811 02:59:00.796702 13690 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/test-user.keytab.
Aug 11 02:59:01 dist-test-slave-p3t5 krb5kdc[21591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:59:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:59:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.94.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.94.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab.
Entry for principal kudu/127.13.94.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab.
I20260811 02:59:01.183748 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:44429
--webserver_interface=127.13.94.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab
--principal=kudu/127.13.94.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.94.148:34273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:44429,127.13.94.189:36825 with env {KRB5CCNAME=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:59:01.560061 21606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:01.560461 21606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:01.560595 21606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:01.568444 21606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:59:01.568635 21606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:01.568722 21606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:59:01.568830 21606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:59:01.579741 21606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:34273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:44429,127.13.94.189:36825
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.190.keytab
--principal=kudu/127.13.94.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.190:44429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.94.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:59:01.580128 21606 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:44429,127.13.94.189:36825'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:59:01.581384 21606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:01.582893 21606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:01.596423 21612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:01.596349 21615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:01.596457 21613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:01.597275 21606 server_base.cc:1034] running on GCE node
I20260811 02:59:01.598256 21606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:01.599901 21606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:01.601132 21606 hybrid_clock.cc:648] HybridClock initialized: now 1786417141601061 us; error 56 us; skew 500 ppm
I20260811 02:59:01.601548 21606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 02:59:01 dist-test-slave-p3t5 krb5kdc[21591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417141, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:01.609431 21606 init.cc:377] Logged in from keytab as kudu/127.13.94.190@KRBTEST.COM (short username kudu)
I20260811 02:59:01.611148 21606 webserver.cc:460] Webserver started at http://127.13.94.190:40893/ using document root <none> and password file <none>
I20260811 02:59:01.611961 21606 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:01.612118 21606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:59:01.612476 21606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:59:01.615567 21606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data/instance:
uuid: "561bec61404c4d85970a94ca3f5af39a"
format_stamp: "Formatted at 2026-08-11 02:59:01 on dist-test-slave-p3t5"
I20260811 02:59:01.616432 21606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/wal/instance:
uuid: "561bec61404c4d85970a94ca3f5af39a"
format_stamp: "Formatted at 2026-08-11 02:59:01 on dist-test-slave-p3t5"
W20260811 02:59:01.622293 21606 dir_manager.cc:174] Insufficient disk space under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data/data: will retry after 10 seconds: IO error: Insufficient disk space to allocate 0 bytes under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data/data (516153344 bytes available vs 518351011 bytes reserved) (error 28)
I20260811 02:59:01.622493 21606 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 02:59:01.626134 21622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:01.627600 21606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:59:01.627811 21606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/wal
uuid: "561bec61404c4d85970a94ca3f5af39a"
format_stamp: "Formatted at 2026-08-11 02:59:01 on dist-test-slave-p3t5"
I20260811 02:59:01.628070 21606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:01.674813 21606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:01.679584 21606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:01.679927 21606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:01.700740 21606 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.190:44429
I20260811 02:59:01.700781 21678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.190:44429 every 8 connection(s)
I20260811 02:59:01.702255 21606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data/info.pb
I20260811 02:59:01.703614 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 21606
I20260811 02:59:01.703848 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/wal/instance
W20260811 02:59:01.707520 21679 dir_manager.cc:174] Insufficient disk space under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data/data: will retry after 10 seconds: IO error: Insufficient disk space to allocate 0 bytes under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-0/data/data (516149248 bytes available vs 518351011 bytes reserved) (error 28)
E20260811 02:59:01.707983 21679 master.cc:392] Unable to init master catalog manager: IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: Failed to create TabletMetadata: All healthy data directories are full (error 28)
I20260811 02:59:01.708258 21606 master.cc:537] Master@127.13.94.190:44429 shutting down...
I20260811 02:59:01.733438 21606 master.cc:552] Master@127.13.94.190:44429 shutdown complete.
IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: Failed to create TabletMetadata: All healthy data directories are full (error 28)
WARNING: no policy specified for kudu/127.13.94.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.94.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.94.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab.
Entry for principal kudu/127.13.94.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab.
I20260811 02:59:01.792888 13690 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
/tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:36825
--webserver_interface=127.13.94.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab
--principal=kudu/127.13.94.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.94.148:34273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.94.190:44429,127.13.94.189:36825 with env {KRB5CCNAME=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:59:02.175436 21684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:02.175791 21684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:02.175979 21684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:02.183804 21684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:59:02.183950 21684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:02.184044 21684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:59:02.184160 21684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:59:02.194743 21684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.94.148:34273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.94.190:44429,127.13.94.189:36825
--keytab_file=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/krb5kdc/kudu_127.13.94.189.keytab
--principal=kudu/127.13.94.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.94.189:36825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.94.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 02:59:02.195219 21684 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.94.190:44429,127.13.94.189:36825'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:59:02.196424 21684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:02.197942 21684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:02.211251 21689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:02.211429 21690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:02.211275 21692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:02.212852 21684 server_base.cc:1034] running on GCE node
I20260811 02:59:02.213891 21684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:02.215612 21684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:02.216892 21684 hybrid_clock.cc:648] HybridClock initialized: now 1786417142216807 us; error 69 us; skew 500 ppm
I20260811 02:59:02.217356 21684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 02:59:02 dist-test-slave-p3t5 krb5kdc[21591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417142, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.94.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:02.221318 21684 init.cc:377] Logged in from keytab as kudu/127.13.94.189@KRBTEST.COM (short username kudu)
I20260811 02:59:02.223217 21684 webserver.cc:460] Webserver started at http://127.13.94.189:40713/ using document root <none> and password file <none>
I20260811 02:59:02.223989 21684 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:02.224100 21684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:59:02.224363 21684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:59:02.227468 21684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data/instance:
uuid: "6ef45c07727e4549b2589688cb1cb4fb"
format_stamp: "Formatted at 2026-08-11 02:59:02 on dist-test-slave-p3t5"
I20260811 02:59:02.228291 21684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/wal/instance:
uuid: "6ef45c07727e4549b2589688cb1cb4fb"
format_stamp: "Formatted at 2026-08-11 02:59:02 on dist-test-slave-p3t5"
W20260811 02:59:02.234474 21684 dir_manager.cc:174] Insufficient disk space under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data/data: will retry after 10 seconds: IO error: Insufficient disk space to allocate 0 bytes under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data/data (516083712 bytes available vs 518351011 bytes reserved) (error 28)
I20260811 02:59:02.234651 21684 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 02:59:02.238405 21699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:02.240051 21684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:59:02.240375 21684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data,/tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/wal
uuid: "6ef45c07727e4549b2589688cb1cb4fb"
format_stamp: "Formatted at 2026-08-11 02:59:02 on dist-test-slave-p3t5"
I20260811 02:59:02.240649 21684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:02.258625 21684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:02.266119 21684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:02.266503 21684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:02.288172 21684 rpc_server.cc:307] RPC server started. Bound to: 127.13.94.189:36825
I20260811 02:59:02.288192 21755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.94.189:36825 every 8 connection(s)
I20260811 02:59:02.289839 21684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data/info.pb
I20260811 02:59:02.293244 13690 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn3Hlw4/build/asan/bin/kudu as pid 21684
I20260811 02:59:02.293504 13690 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/wal/instance
W20260811 02:59:02.294257 21756 dir_manager.cc:174] Insufficient disk space under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data/data: will retry after 10 seconds: IO error: Insufficient disk space to allocate 0 bytes under path /tmp/dist-test-taskn3Hlw4/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786417032720145-13690-0/minicluster-data/master-1/data/data (516079616 bytes available vs 518351011 bytes reserved) (error 28)
E20260811 02:59:02.294616 21756 master.cc:392] Unable to init master catalog manager: IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: Failed to create TabletMetadata: All healthy data directories are full (error 28)
I20260811 02:59:02.294862 21684 master.cc:537] Master@127.13.94.189:36825 shutting down...
Aug 11 02:59:02 dist-test-slave-p3t5 krb5kdc[21591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.94.189@KRBTEST.COM
W20260811 02:59:02.323676 21702 connection.cc:441] server connection from 127.0.0.1:49646 torn down before Call kudu.master.MasterService.ConnectToMaster from 127.0.0.1:49646 (request call id 0) could send its response
I20260811 02:59:02.337220 21684 master.cc:552] Master@127.13.94.189:36825 shutdown complete.
IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: Failed to create TabletMetadata: All healthy data directories are full (error 28)
F20260811 02:59:07.372795 13690 external_mini_cluster.cc:634] Check failed: leader_master() != nullptr Must have started at least 1 master before adding tablet servers
*** Check failure stack trace: ***
*** Aborted at 1786417147 (unix time) try "date -d @1786417147" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e80000357a) received by PID 13690 (TID 0x7fde6b42ad40) from PID 13690; stack trace: ***
@ 0x7fde6eb76a46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
@ 0x7fde76bc6980 (unknown) at ??:0
@ 0x7fde6c11ae87 gsignal at ??:0
@ 0x7fde6c11c7f1 abort at ??:0
@ 0x7fde6eb689ad google::LogMessage::Fail() at ??:0
@ 0x7fde6eb6b88a google::LogMessage::SendToLog() at ??:0
@ 0x7fde6eb684de google::LogMessage::Flush() at ??:0
@ 0x7fde6eb69a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fde845bcbc5 kudu::cluster::ExternalMiniCluster::AddTabletServer() at ??:0
@ 0x7fde845b63f0 kudu::cluster::ExternalMiniCluster::Start() at ??:0
@ 0x55506a kudu::master::AutoAddMasterTest::SetUpWithTestArgs() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/dynamic_multi_master-test.cc:1466
@ 0x555aa1 kudu::master::ParameterizedAutoAddMasterTest::SetUp() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/dynamic_multi_master-test.cc:1824
@ 0x7fde70985bfe testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
@ 0x7fde7097a566 testing::Test::Run() at ??:0
@ 0x7fde7097a795 testing::TestInfo::Run() at ??:0
@ 0x7fde7097acdf testing::TestSuite::Run() at ??:0
@ 0x7fde7097b29f testing::internal::UnitTestImpl::RunAllTests() at ??:0
@ 0x7fde7098613e testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
@ 0x7fde7097a83d testing::UnitTest::Run() at ??:0
@ 0x5d3fa2 RUN_ALL_TESTS() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/gtest/gtest.h:2293
@ 0x5d24b2 main at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:109
@ 0x7fde6c0fdc87 __libc_start_main at ??:0
@ 0x413329 (unknown) at ??:?