[==========] Running 6 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from RaftConfigChangeITest
[ RUN ] RaftConfigChangeITest.TestKudu2147
2026-11-15T21:02:06Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-15T21:02:06Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20261115 21:02:06.037911 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:37115
--webserver_interface=127.29.135.254
--webserver_port=0
--builtin_ntp_servers=127.29.135.212:44305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.135.254:37115 with env {}
W20261115 21:02:06.131650 30247 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:06.131786 30247 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:06.131822 30247 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:06.135074 30247 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261115 21:02:06.135123 30247 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:06.135142 30247 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261115 21:02:06.135164 30247 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261115 21:02:06.138885 30247 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:44305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.135.254:37115
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:37115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.135.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.30247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:06.139539 30247 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:06.140161 30247 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:06.144636 30252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:06.144747 30247 server_base.cc:1061] running on GCE node
W20261115 21:02:06.144815 30255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:06.144649 30253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:06.145239 30247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:06.145865 30247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:06.146996 30247 hybrid_clock.cc:648] HybridClock initialized: now 1794776526146981 us; error 27 us; skew 500 ppm
I20261115 21:02:06.148387 30247 webserver.cc:577] Webserver started at http://127.29.135.254:35741/ using document root <none> and password file <none>
I20261115 21:02:06.148869 30247 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:06.148910 30247 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:06.149077 30247 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:06.150242 30247 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/data/instance:
uuid: "fd786ae6a14b46d2b36e2ea62885056f"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.150653 30247 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/wal/instance:
uuid: "fd786ae6a14b46d2b36e2ea62885056f"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.153445 30247 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20261115 21:02:06.155397 30261 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:06.156225 30247 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:06.156353 30247 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/wal
uuid: "fd786ae6a14b46d2b36e2ea62885056f"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.156427 30247 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:06.172698 30247 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:06.173071 30247 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:06.173190 30247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:06.179489 30247 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.254:37115
I20261115 21:02:06.179494 30313 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.254:37115 every 8 connection(s)
I20261115 21:02:06.180374 30247 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/data/info.pb
I20261115 21:02:06.181929 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 30247
I20261115 21:02:06.182054 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/master-0/wal/instance
I20261115 21:02:06.183220 30314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:06.188444 30314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f: Bootstrap starting.
I20261115 21:02:06.190407 30314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:06.191109 30314 log.cc:826] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:06.192458 30314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f: No bootstrap required, opened a new log
I20261115 21:02:06.194674 30314 raft_consensus.cc:360] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd786ae6a14b46d2b36e2ea62885056f" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 37115 } }
I20261115 21:02:06.194846 30314 raft_consensus.cc:386] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:06.194885 30314 raft_consensus.cc:748] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd786ae6a14b46d2b36e2ea62885056f, State: Initialized, Role: FOLLOWER
I20261115 21:02:06.195228 30314 consensus_queue.cc:261] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd786ae6a14b46d2b36e2ea62885056f" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 37115 } }
I20261115 21:02:06.195345 30314 raft_consensus.cc:400] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261115 21:02:06.195386 30314 raft_consensus.cc:499] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261115 21:02:06.195443 30314 raft_consensus.cc:3096] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:06.196063 30314 raft_consensus.cc:521] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd786ae6a14b46d2b36e2ea62885056f" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 37115 } }
I20261115 21:02:06.196348 30314 leader_election.cc:304] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd786ae6a14b46d2b36e2ea62885056f; no voters:
I20261115 21:02:06.196554 30314 leader_election.cc:290] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [CANDIDATE]: Term 1 election: Requested vote from peers
I20261115 21:02:06.196637 30319 raft_consensus.cc:2826] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:06.196823 30319 raft_consensus.cc:705] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [term 1 LEADER]: Becoming Leader. State: Replica: fd786ae6a14b46d2b36e2ea62885056f, State: Running, Role: LEADER
I20261115 21:02:06.197124 30319 consensus_queue.cc:238] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd786ae6a14b46d2b36e2ea62885056f" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 37115 } }
I20261115 21:02:06.197465 30314 sys_catalog.cc:565] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [sys.catalog]: configured and running, proceeding with master startup.
I20261115 21:02:06.198477 30321 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd786ae6a14b46d2b36e2ea62885056f. Latest consensus state: current_term: 1 leader_uuid: "fd786ae6a14b46d2b36e2ea62885056f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd786ae6a14b46d2b36e2ea62885056f" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 37115 } } }
I20261115 21:02:06.198616 30321 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:06.198868 30320 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd786ae6a14b46d2b36e2ea62885056f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd786ae6a14b46d2b36e2ea62885056f" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 37115 } } }
I20261115 21:02:06.198972 30320 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:06.199643 30328 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261115 21:02:06.202822 30328 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261115 21:02:06.207695 30328 catalog_manager.cc:1398] Generated new cluster ID: 4c0403f1e235452494a06f9e7a058d7c
I20261115 21:02:06.207759 30328 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261115 21:02:06.234193 30328 catalog_manager.cc:1421] Generated new certificate authority record
I20261115 21:02:06.235210 30328 catalog_manager.cc:1555] Loading token signing keys...
I20261115 21:02:06.251698 30328 catalog_manager.cc:6221] T 00000000000000000000000000000000 P fd786ae6a14b46d2b36e2ea62885056f: Generated new TSK 0
I20261115 21:02:06.252310 30328 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261115 21:02:06.259470 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.193:0
--local_ip_for_outbound_sockets=127.29.135.193
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:37115
--builtin_ntp_servers=127.29.135.212:44305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000 with env {}
W20261115 21:02:06.353268 30338 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:06.353402 30338 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:06.353457 30338 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:06.356671 30338 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:06.356782 30338 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.193
I20261115 21:02:06.360525 30338 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:44305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:37115
--never_fsync=true
--heap_profile_path=/tmp/kudu.30338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:06.361153 30338 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:06.361783 30338 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:06.366664 30346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:06.366671 30343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:06.366671 30344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:06.366746 30338 server_base.cc:1061] running on GCE node
I20261115 21:02:06.367139 30338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:06.367600 30338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:06.368731 30338 hybrid_clock.cc:648] HybridClock initialized: now 1794776526368715 us; error 30 us; skew 500 ppm
I20261115 21:02:06.370060 30338 webserver.cc:577] Webserver started at http://127.29.135.193:44589/ using document root <none> and password file <none>
I20261115 21:02:06.370561 30338 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:06.370611 30338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:06.370806 30338 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:06.371970 30338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/data/instance:
uuid: "705537d609d64a9b97fa268d3ae5107c"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.372344 30338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/wal/instance:
uuid: "705537d609d64a9b97fa268d3ae5107c"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.375214 30338 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20261115 21:02:06.377103 30352 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:06.377929 30338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20261115 21:02:06.378022 30338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/wal
uuid: "705537d609d64a9b97fa268d3ae5107c"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.378093 30338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:06.402034 30338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:06.402408 30338 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:06.402546 30338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:06.402971 30338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:06.403795 30338 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:06.403847 30338 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:06.403898 30338 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:06.403926 30338 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:06.411649 30338 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.193:40087
I20261115 21:02:06.411689 30465 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.193:40087 every 8 connection(s)
I20261115 21:02:06.412484 30338 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/data/info.pb
I20261115 21:02:06.413820 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 30338
I20261115 21:02:06.413925 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-0/wal/instance
I20261115 21:02:06.416045 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.194:0
--local_ip_for_outbound_sockets=127.29.135.194
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:37115
--builtin_ntp_servers=127.29.135.212:44305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000 with env {}
I20261115 21:02:06.419950 30466 heartbeater.cc:345] Connected to a master server at 127.29.135.254:37115
I20261115 21:02:06.420140 30466 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:06.420555 30466 heartbeater.cc:508] Master 127.29.135.254:37115 requested a full tablet report, sending...
I20261115 21:02:06.421964 30278 ts_manager.cc:210] Registered new tserver with Master: 705537d609d64a9b97fa268d3ae5107c (127.29.135.193:40087)
I20261115 21:02:06.423179 30278 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.193:53667
W20261115 21:02:06.511898 30469 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:06.512032 30469 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:06.512090 30469 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:06.515311 30469 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:06.515430 30469 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.194
I20261115 21:02:06.519129 30469 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:44305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:37115
--never_fsync=true
--heap_profile_path=/tmp/kudu.30469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:06.519804 30469 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:06.520448 30469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:06.525606 30475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:06.525612 30477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:06.525794 30474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:06.526129 30469 server_base.cc:1061] running on GCE node
I20261115 21:02:06.526516 30469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:06.527170 30469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:06.528357 30469 hybrid_clock.cc:648] HybridClock initialized: now 1794776526528342 us; error 30 us; skew 500 ppm
I20261115 21:02:06.529755 30469 webserver.cc:577] Webserver started at http://127.29.135.194:34817/ using document root <none> and password file <none>
I20261115 21:02:06.530267 30469 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:06.530316 30469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:06.530543 30469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:06.531817 30469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/data/instance:
uuid: "3363b0ca6a314e4a90dacc5841e9a51f"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.532223 30469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/wal/instance:
uuid: "3363b0ca6a314e4a90dacc5841e9a51f"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.535226 30469 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20261115 21:02:06.537305 30483 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:06.538190 30469 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:06.538288 30469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/wal
uuid: "3363b0ca6a314e4a90dacc5841e9a51f"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.538367 30469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:06.548023 30469 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:06.548419 30469 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:06.548549 30469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:06.548979 30469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:06.549825 30469 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:06.549870 30469 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:06.549921 30469 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:06.549949 30469 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:06.557760 30469 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.194:33685
I20261115 21:02:06.557778 30596 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.194:33685 every 8 connection(s)
I20261115 21:02:06.558633 30469 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/data/info.pb
I20261115 21:02:06.560652 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 30469
I20261115 21:02:06.560755 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/wal/instance
I20261115 21:02:06.562826 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.195:0
--local_ip_for_outbound_sockets=127.29.135.195
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:37115
--builtin_ntp_servers=127.29.135.212:44305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000 with env {}
I20261115 21:02:06.565937 30597 heartbeater.cc:345] Connected to a master server at 127.29.135.254:37115
I20261115 21:02:06.566160 30597 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:06.566725 30597 heartbeater.cc:508] Master 127.29.135.254:37115 requested a full tablet report, sending...
I20261115 21:02:06.567771 30278 ts_manager.cc:210] Registered new tserver with Master: 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194:33685)
I20261115 21:02:06.568315 30278 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.194:47869
W20261115 21:02:06.660394 30600 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:06.660537 30600 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:06.660584 30600 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:06.663952 30600 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:06.664064 30600 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.195
I20261115 21:02:06.667853 30600 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:44305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:37115
--never_fsync=true
--heap_profile_path=/tmp/kudu.30600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:06.668583 30600 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:06.669246 30600 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:06.674579 30606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:06.674592 30608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:06.674579 30605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:06.675105 30600 server_base.cc:1061] running on GCE node
I20261115 21:02:06.675414 30600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:06.675873 30600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:06.677011 30600 hybrid_clock.cc:648] HybridClock initialized: now 1794776526676994 us; error 30 us; skew 500 ppm
I20261115 21:02:06.678427 30600 webserver.cc:577] Webserver started at http://127.29.135.195:33847/ using document root <none> and password file <none>
I20261115 21:02:06.678973 30600 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:06.679024 30600 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:06.679219 30600 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:06.680431 30600 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/data/instance:
uuid: "a8f9e0e8fe36443ea785a79d732b1254"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.680853 30600 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/wal/instance:
uuid: "a8f9e0e8fe36443ea785a79d732b1254"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.683856 30600 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20261115 21:02:06.685823 30614 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:06.686697 30600 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20261115 21:02:06.686805 30600 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/wal
uuid: "a8f9e0e8fe36443ea785a79d732b1254"
format_stamp: "Formatted at 2026-11-15 21:02:06 on dist-test-slave-xh31"
I20261115 21:02:06.686880 30600 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:06.716259 30600 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:06.716722 30600 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:06.716861 30600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:06.717361 30600 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:06.718315 30600 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:06.718362 30600 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:06.718420 30600 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:06.718443 30600 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:06.726714 30600 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.195:35377
I20261115 21:02:06.726748 30727 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.195:35377 every 8 connection(s)
I20261115 21:02:06.727743 30600 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/data/info.pb
I20261115 21:02:06.734372 30728 heartbeater.cc:345] Connected to a master server at 127.29.135.254:37115
I20261115 21:02:06.734584 30728 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:06.735010 30728 heartbeater.cc:508] Master 127.29.135.254:37115 requested a full tablet report, sending...
I20261115 21:02:06.735905 30278 ts_manager.cc:210] Registered new tserver with Master: a8f9e0e8fe36443ea785a79d732b1254 (127.29.135.195:35377)
I20261115 21:02:06.736423 30278 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.195:47453
I20261115 21:02:06.737711 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 30600
I20261115 21:02:06.737793 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-2/wal/instance
I20261115 21:02:06.739980 30239 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20261115 21:02:06.758119 30278 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47296:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20261115 21:02:06.759160 30278 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261115 21:02:06.773873 30662 tablet_service.cc:1500] Processing CreateTablet for tablet 62621c676602415f801765f5fff79b0f (DEFAULT_TABLE table=test-workload [id=04cb3b19371c47a2a341c2e86b414ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:06.773916 30400 tablet_service.cc:1500] Processing CreateTablet for tablet 62621c676602415f801765f5fff79b0f (DEFAULT_TABLE table=test-workload [id=04cb3b19371c47a2a341c2e86b414ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:06.773916 30531 tablet_service.cc:1500] Processing CreateTablet for tablet 62621c676602415f801765f5fff79b0f (DEFAULT_TABLE table=test-workload [id=04cb3b19371c47a2a341c2e86b414ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:06.774741 30662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62621c676602415f801765f5fff79b0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:06.774785 30400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62621c676602415f801765f5fff79b0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:06.774794 30531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62621c676602415f801765f5fff79b0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:06.779613 30750 tablet_bootstrap.cc:492] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254: Bootstrap starting.
I20261115 21:02:06.780578 30752 tablet_bootstrap.cc:492] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Bootstrap starting.
I20261115 21:02:06.781829 30750 tablet_bootstrap.cc:654] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:06.782546 30752 tablet_bootstrap.cc:654] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:06.782692 30750 log.cc:826] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:06.782867 30751 tablet_bootstrap.cc:492] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c: Bootstrap starting.
I20261115 21:02:06.783370 30752 log.cc:826] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:06.784060 30750 tablet_bootstrap.cc:492] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254: No bootstrap required, opened a new log
I20261115 21:02:06.784245 30750 ts_tablet_manager.cc:1449] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20261115 21:02:06.784761 30752 tablet_bootstrap.cc:492] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: No bootstrap required, opened a new log
I20261115 21:02:06.784964 30751 tablet_bootstrap.cc:654] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:06.784963 30752 ts_tablet_manager.cc:1449] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20261115 21:02:06.785763 30751 log.cc:826] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:06.786788 30750 raft_consensus.cc:360] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 } } peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:06.786955 30750 raft_consensus.cc:386] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:06.787011 30750 raft_consensus.cc:748] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8f9e0e8fe36443ea785a79d732b1254, State: Initialized, Role: FOLLOWER
I20261115 21:02:06.787143 30751 tablet_bootstrap.cc:492] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c: No bootstrap required, opened a new log
I20261115 21:02:06.787328 30751 ts_tablet_manager.cc:1449] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20261115 21:02:06.787421 30750 consensus_queue.cc:261] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 } } peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:06.787942 30752 raft_consensus.cc:360] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 } } peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:06.788137 30752 raft_consensus.cc:386] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:06.788184 30750 ts_tablet_manager.cc:1480] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20261115 21:02:06.788231 30752 raft_consensus.cc:748] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3363b0ca6a314e4a90dacc5841e9a51f, State: Initialized, Role: FOLLOWER
I20261115 21:02:06.788545 30728 heartbeater.cc:500] Master 127.29.135.254:37115 was elected leader, sending a full tablet report...
I20261115 21:02:06.789742 30752 consensus_queue.cc:261] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 } } peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:06.790198 30751 raft_consensus.cc:360] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 } } peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:06.790367 30597 heartbeater.cc:500] Master 127.29.135.254:37115 was elected leader, sending a full tablet report...
I20261115 21:02:06.790441 30751 raft_consensus.cc:386] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:06.790516 30751 raft_consensus.cc:748] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 705537d609d64a9b97fa268d3ae5107c, State: Initialized, Role: FOLLOWER
I20261115 21:02:06.790593 30752 ts_tablet_manager.cc:1480] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20261115 21:02:06.790911 30751 consensus_queue.cc:261] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 } } peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:06.791429 30466 heartbeater.cc:500] Master 127.29.135.254:37115 was elected leader, sending a full tablet report...
I20261115 21:02:06.791543 30751 ts_tablet_manager.cc:1480] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20261115 21:02:06.809386 30598 tablet.cc:2441] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261115 21:02:06.913422 30467 tablet.cc:2441] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261115 21:02:06.978565 30729 tablet.cc:2441] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261115 21:02:07.756587 30759 raft_consensus.cc:499] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261115 21:02:07.756757 30759 raft_consensus.cc:521] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 } } peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:07.757844 30759 leader_election.cc:290] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194:33685), a8f9e0e8fe36443ea785a79d732b1254 (127.29.135.195:35377)
I20261115 21:02:07.762234 30551 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "62621c676602415f801765f5fff79b0f" candidate_uuid: "705537d609d64a9b97fa268d3ae5107c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" is_pre_election: true
I20261115 21:02:07.762243 30682 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "62621c676602415f801765f5fff79b0f" candidate_uuid: "705537d609d64a9b97fa268d3ae5107c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8f9e0e8fe36443ea785a79d732b1254" is_pre_election: true
I20261115 21:02:07.762477 30551 raft_consensus.cc:2490] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 705537d609d64a9b97fa268d3ae5107c in term 0.
I20261115 21:02:07.762526 30682 raft_consensus.cc:2490] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 705537d609d64a9b97fa268d3ae5107c in term 0.
I20261115 21:02:07.762914 30353 leader_election.cc:304] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3363b0ca6a314e4a90dacc5841e9a51f, 705537d609d64a9b97fa268d3ae5107c; no voters:
I20261115 21:02:07.763123 30759 raft_consensus.cc:2826] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261115 21:02:07.763185 30759 raft_consensus.cc:499] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261115 21:02:07.763216 30759 raft_consensus.cc:3096] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:07.764076 30759 raft_consensus.cc:521] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 } } peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:07.764389 30759 leader_election.cc:290] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [CANDIDATE]: Term 1 election: Requested vote from peers 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194:33685), a8f9e0e8fe36443ea785a79d732b1254 (127.29.135.195:35377)
I20261115 21:02:07.764669 30682 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "62621c676602415f801765f5fff79b0f" candidate_uuid: "705537d609d64a9b97fa268d3ae5107c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8f9e0e8fe36443ea785a79d732b1254"
I20261115 21:02:07.764694 30551 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "62621c676602415f801765f5fff79b0f" candidate_uuid: "705537d609d64a9b97fa268d3ae5107c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3363b0ca6a314e4a90dacc5841e9a51f"
I20261115 21:02:07.764789 30551 raft_consensus.cc:3096] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:07.764788 30682 raft_consensus.cc:3096] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:07.765558 30682 raft_consensus.cc:2490] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 705537d609d64a9b97fa268d3ae5107c in term 1.
I20261115 21:02:07.765657 30551 raft_consensus.cc:2490] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 705537d609d64a9b97fa268d3ae5107c in term 1.
I20261115 21:02:07.765817 30356 leader_election.cc:304] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 705537d609d64a9b97fa268d3ae5107c, a8f9e0e8fe36443ea785a79d732b1254; no voters:
I20261115 21:02:07.766000 30759 raft_consensus.cc:2826] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:07.766160 30759 raft_consensus.cc:705] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 1 LEADER]: Becoming Leader. State: Replica: 705537d609d64a9b97fa268d3ae5107c, State: Running, Role: LEADER
I20261115 21:02:07.766507 30759 consensus_queue.cc:238] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 } } peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:07.769565 30766 consensus_queue.cc:1049] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:07.769716 30764 consensus_queue.cc:1049] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:07.770139 30277 catalog_manager.cc:5840] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c reported cstate change: term changed from 0 to 1, leader changed from <none> to 705537d609d64a9b97fa268d3ae5107c (127.29.135.193). New cstate: current_term: 1 leader_uuid: "705537d609d64a9b97fa268d3ae5107c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 33685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } health_report { overall_health: UNKNOWN } } }
W20261115 21:02:08.005064 30460 generic_service.cc:139] {username='slave'} at 127.0.0.1:36228 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261115 21:02:08.005173 30460 generic_service.cc:175] {username='slave'} at 127.0.0.1:36228 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'false'
I20261115 21:02:08.005945 30460 generic_service.cc:175] {username='slave'} at 127.0.0.1:36228 changed flags via RPC: heartbeat_inject_latency_before_heartbeat_ms from '0' to '2000'
W20261115 21:02:08.006948 30591 generic_service.cc:139] {username='slave'} at 127.0.0.1:41790 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261115 21:02:08.007041 30591 generic_service.cc:175] {username='slave'} at 127.0.0.1:41790 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'false'
I20261115 21:02:08.007740 30591 generic_service.cc:175] {username='slave'} at 127.0.0.1:41790 changed flags via RPC: heartbeat_inject_latency_before_heartbeat_ms from '0' to '5000'
W20261115 21:02:08.008780 30722 generic_service.cc:139] {username='slave'} at 127.0.0.1:53350 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261115 21:02:08.008865 30722 generic_service.cc:175] {username='slave'} at 127.0.0.1:53350 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'false'
I20261115 21:02:08.009496 30722 generic_service.cc:175] {username='slave'} at 127.0.0.1:53350 changed flags via RPC: heartbeat_inject_latency_before_heartbeat_ms from '0' to '5000'
I20261115 21:02:08.010881 30420 consensus_queue.cc:238] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 1.34, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:08.011888 30681 raft_consensus.cc:1285] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 1 FOLLOWER]: Refusing update from remote peer 705537d609d64a9b97fa268d3ae5107c: Log matching property violated. Preceding OpId in replica: term: 1 index: 34. Preceding OpId from leader: term: 1 index: 35. (index mismatch)
I20261115 21:02:08.012211 30766 consensus_queue.cc:1049] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20261115 21:02:08.013675 30759 raft_consensus.cc:2983] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 1 LEADER]: Committing config change with OpId 1.35: config changed from index -1 to 35, VOTER 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } } }
I20261115 21:02:08.014066 30681 raft_consensus.cc:2983] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 1 FOLLOWER]: Committing config change with OpId 1.35: config changed from index -1 to 35, VOTER 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } } }
W20261115 21:02:08.015986 30460 generic_service.cc:139] {username='slave'} at 127.0.0.1:36228 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261115 21:02:08.016055 30460 generic_service.cc:175] {username='slave'} at 127.0.0.1:36228 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'true'
I20261115 21:02:08.016706 30681 tablet_service.cc:2033] Received Run Leader Election RPC: tablet_id: "62621c676602415f801765f5fff79b0f"
dest_uuid: "a8f9e0e8fe36443ea785a79d732b1254"
from {username='slave'} at 127.0.0.1:53350
I20261115 21:02:08.016820 30681 raft_consensus.cc:499] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20261115 21:02:08.016868 30681 raft_consensus.cc:3096] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 1 FOLLOWER]: Advancing to term 2
I20261115 21:02:08.017443 30681 raft_consensus.cc:521] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:08.018195 30681 leader_election.cc:290] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [CANDIDATE]: Term 2 election: Requested vote from peers 705537d609d64a9b97fa268d3ae5107c (127.29.135.193:40087)
I20261115 21:02:08.021562 30420 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "62621c676602415f801765f5fff79b0f" candidate_uuid: "a8f9e0e8fe36443ea785a79d732b1254" candidate_term: 2 candidate_status { last_received { term: 1 index: 35 } } ignore_live_leader: true dest_uuid: "705537d609d64a9b97fa268d3ae5107c"
I20261115 21:02:08.021730 30420 raft_consensus.cc:3091] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 1 LEADER]: Stepping down as leader of term 1
I20261115 21:02:08.021780 30420 raft_consensus.cc:748] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 705537d609d64a9b97fa268d3ae5107c, State: Running, Role: LEADER
I20261115 21:02:08.021893 30420 consensus_queue.cc:261] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 1.35, Last appended by leader: 35, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
I20261115 21:02:08.022001 30420 raft_consensus.cc:3096] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 1 FOLLOWER]: Advancing to term 2
I20261115 21:02:08.022763 30420 raft_consensus.cc:2490] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8f9e0e8fe36443ea785a79d732b1254 in term 2.
I20261115 21:02:08.023095 30616 leader_election.cc:304] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 705537d609d64a9b97fa268d3ae5107c, a8f9e0e8fe36443ea785a79d732b1254; no voters:
I20261115 21:02:08.023306 30768 raft_consensus.cc:2826] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 2 FOLLOWER]: Leader election won for term 2
I20261115 21:02:08.023488 30768 raft_consensus.cc:705] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 2 LEADER]: Becoming Leader. State: Replica: a8f9e0e8fe36443ea785a79d732b1254, State: Running, Role: LEADER
I20261115 21:02:08.023762 30768 consensus_queue.cc:238] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 1.35, Last appended by leader: 35, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } }
W20261115 21:02:08.025504 30616 consensus_peers.cc:597] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 -> Peer 705537d609d64a9b97fa268d3ae5107c (127.29.135.193:40087): Couldn't send request to peer 705537d609d64a9b97fa268d3ae5107c. Error code: UNKNOWN_ERROR (1). Status: Illegal state: Rejected: --follower_reject_update_consensus_requests is set to true.. This is attempt 1: this message will repeat every 5th retry.
I20261115 21:02:10.016670 30271 catalog_manager.cc:5840] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c reported cstate change: config changed from index -1 to 35, term changed from 1 to 2, leader changed from 705537d609d64a9b97fa268d3ae5107c (127.29.135.193) to <none>, VOTER 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194) evicted. New cstate: current_term: 2 committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } } }
I20261115 21:02:10.019908 30531 tablet_service.cc:1547] Processing DeleteTablet for tablet 62621c676602415f801765f5fff79b0f with delete_type TABLET_DATA_TOMBSTONED (TS 3363b0ca6a314e4a90dacc5841e9a51f not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:41774
I20261115 21:02:10.020417 30804 tablet_replica.cc:333] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: stopping tablet replica
I20261115 21:02:10.020668 30804 raft_consensus.cc:2261] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261115 21:02:10.020888 30804 raft_consensus.cc:2290] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261115 21:02:10.022886 30804 ts_tablet_manager.cc:1962] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20261115 21:02:10.024672 30804 ts_tablet_manager.cc:1975] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.34
I20261115 21:02:10.024746 30804 log.cc:1199] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Deleting WAL directory at /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794776526014215-30239-0/minicluster-data/ts-1/wal/wals/62621c676602415f801765f5fff79b0f
I20261115 21:02:10.025280 30262 catalog_manager.cc:5065] TS 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194:33685): tablet 62621c676602415f801765f5fff79b0f (table test-workload [id=04cb3b19371c47a2a341c2e86b414ef0]) successfully deleted
W20261115 21:02:10.058389 30460 generic_service.cc:139] {username='slave'} at 127.0.0.1:36228 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261115 21:02:10.058483 30460 generic_service.cc:175] {username='slave'} at 127.0.0.1:36228 changed flags via RPC: follower_reject_update_consensus_requests from 'true' to 'false'
I20261115 21:02:10.061036 30239 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 2.36 1.35
I20261115 21:02:10.163805 30239 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 2.36 1.35
I20261115 21:02:10.366262 30239 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 2.36 1.35
I20261115 21:02:10.668720 30239 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 2.36 1.35
I20261115 21:02:11.071235 30239 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 2.36 1.35
I20261115 21:02:11.574661 30239 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 2.36 2.36
I20261115 21:02:12.021220 30271 catalog_manager.cc:5840] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c reported cstate change: leader changed from <none> to a8f9e0e8fe36443ea785a79d732b1254 (127.29.135.195). New cstate: current_term: 2 leader_uuid: "a8f9e0e8fe36443ea785a79d732b1254" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } } }
I20261115 21:02:12.177078 30239 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 2.36 2.36
I20261115 21:02:12.879786 30239 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 2.36 2.36
I20261115 21:02:13.019634 30681 consensus_queue.cc:238] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 2.36, Last appended by leader: 35, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } } peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 33685 } attrs { promote: true } }
I20261115 21:02:13.021323 30420 raft_consensus.cc:1285] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 2 FOLLOWER]: Refusing update from remote peer a8f9e0e8fe36443ea785a79d732b1254: Log matching property violated. Preceding OpId in replica: term: 2 index: 36. Preceding OpId from leader: term: 2 index: 37. (index mismatch)
I20261115 21:02:13.021827 30818 consensus_queue.cc:1049] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [LEADER]: Connected to new peer: Peer: permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20261115 21:02:13.023940 30817 raft_consensus.cc:2983] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 [term 2 LEADER]: Committing config change with OpId 2.37: config changed from index 35 to 37, NON_VOTER 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194) added. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } } peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 33685 } attrs { promote: true } } }
I20261115 21:02:13.024379 30420 raft_consensus.cc:2983] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c [term 2 FOLLOWER]: Committing config change with OpId 2.37: config changed from index 35 to 37, NON_VOTER 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194) added. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } } peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 33685 } attrs { promote: true } } }
W20261115 21:02:13.024724 30615 consensus_peers.cc:597] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254 -> Peer 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194:33685): Couldn't send request to peer 3363b0ca6a314e4a90dacc5841e9a51f. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20261115 21:02:13.025699 30265 catalog_manager.cc:5247] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 62621c676602415f801765f5fff79b0f with cas_config_opid_index 35: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20261115 21:02:13.682392 30239 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 2.37 2.37
I20261115 21:02:14.025648 30271 catalog_manager.cc:5840] T 62621c676602415f801765f5fff79b0f P 705537d609d64a9b97fa268d3ae5107c reported cstate change: config changed from index 35 to 37, NON_VOTER 3363b0ca6a314e4a90dacc5841e9a51f (127.29.135.194) added. New cstate: current_term: 2 leader_uuid: "a8f9e0e8fe36443ea785a79d732b1254" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } } peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 33685 } attrs { promote: true } } }
I20261115 21:02:14.584677 30239 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 2.37 2.37
I20261115 21:02:15.368530 30828 ts_tablet_manager.cc:979] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Initiating tablet copy from peer a8f9e0e8fe36443ea785a79d732b1254 (127.29.135.195:35377)
I20261115 21:02:15.369361 30828 tablet_copy_client.cc:323] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: tablet copy: Beginning tablet copy session from remote peer at address 127.29.135.195:35377
I20261115 21:02:15.374743 30702 tablet_copy_service.cc:140] P a8f9e0e8fe36443ea785a79d732b1254: Received BeginTabletCopySession request for tablet 62621c676602415f801765f5fff79b0f from peer 3363b0ca6a314e4a90dacc5841e9a51f ({username='slave'} at 127.29.135.194:47345)
I20261115 21:02:15.374992 30702 tablet_copy_service.cc:161] P a8f9e0e8fe36443ea785a79d732b1254: Beginning new tablet copy session on tablet 62621c676602415f801765f5fff79b0f from peer 3363b0ca6a314e4a90dacc5841e9a51f at {username='slave'} at 127.29.135.194:47345: session id = 3363b0ca6a314e4a90dacc5841e9a51f-62621c676602415f801765f5fff79b0f
I20261115 21:02:15.376768 30702 tablet_copy_source_session.cc:215] T 62621c676602415f801765f5fff79b0f P a8f9e0e8fe36443ea785a79d732b1254: Tablet Copy: opened 0 blocks and 1 log segments
I20261115 21:02:15.378465 30828 ts_tablet_manager.cc:1962] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Deleting tablet data with delete state TABLET_DATA_COPYING
I20261115 21:02:15.380612 30828 ts_tablet_manager.cc:1975] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.34
I20261115 21:02:15.381145 30828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62621c676602415f801765f5fff79b0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:15.382397 30828 tablet_copy_client.cc:806] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: tablet copy: Starting download of 0 data blocks...
I20261115 21:02:15.382613 30828 tablet_copy_client.cc:670] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: tablet copy: Starting download of 1 WAL segments...
I20261115 21:02:15.384826 30828 tablet_copy_client.cc:538] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261115 21:02:15.386361 30828 tablet_bootstrap.cc:492] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Bootstrap starting.
I20261115 21:02:15.441341 30828 tablet_bootstrap.cc:492] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Bootstrap replayed 1/1 log segments. Stats: ops{read=37 overwritten=0 applied=37 ignored=0} inserts{seen=1650 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261115 21:02:15.441661 30828 tablet_bootstrap.cc:492] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Bootstrap complete.
I20261115 21:02:15.441841 30828 ts_tablet_manager.cc:1449] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Time spent bootstrapping tablet: real 0.056s user 0.053s sys 0.001s
I20261115 21:02:15.442342 30828 raft_consensus.cc:360] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } } peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 33685 } attrs { promote: true } }
I20261115 21:02:15.442451 30828 raft_consensus.cc:748] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3363b0ca6a314e4a90dacc5841e9a51f, State: Initialized, Role: LEARNER
I20261115 21:02:15.442579 30828 consensus_queue.cc:261] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 37, Last appended: 2.37, Last appended by leader: 37, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "705537d609d64a9b97fa268d3ae5107c" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40087 } } peers { permanent_uuid: "a8f9e0e8fe36443ea785a79d732b1254" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 35377 } } peers { permanent_uuid: "3363b0ca6a314e4a90dacc5841e9a51f" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 33685 } attrs { promote: true } }
I20261115 21:02:15.442906 30828 ts_tablet_manager.cc:1480] T 62621c676602415f801765f5fff79b0f P 3363b0ca6a314e4a90dacc5841e9a51f: Time spent starting tablet: real 0.001s user 0.000s sys 0.004s
I20261115 21:02:15.443627 30702 tablet_copy_service.cc:342] P a8f9e0e8fe36443ea785a79d732b1254: Request end of tablet copy session 3363b0ca6a314e4a90dacc5841e9a51f-62621c676602415f801765f5fff79b0f received from {username='slave'} at 127.29.135.194:47345
I20261115 21:02:15.443728 30702 tablet_copy_service.cc:434] P a8f9e0e8fe36443ea785a79d732b1254: ending tablet copy session 3363b0ca6a314e4a90dacc5841e9a51f-62621c676602415f801765f5fff79b0f on tablet 62621c676602415f801765f5fff79b0f with peer 3363b0ca6a314e4a90dacc5841e9a51f
I20261115 21:02:15.588248 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 30338
I20261115 21:02:15.593974 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 30469
I20261115 21:02:15.599498 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 30600
I20261115 21:02:15.604626 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 30247
2026-11-15T21:02:15Z chronyd exiting
[ OK ] RaftConfigChangeITest.TestKudu2147 (9601 ms)
[ RUN ] RaftConfigChangeITest.TestNonVoterPromotion
2026-11-15T21:02:15Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-15T21:02:15Z Disabled control of system clock
I20261115 21:02:15.632602 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:40143
--webserver_interface=127.29.135.254
--webserver_port=0
--builtin_ntp_servers=127.29.135.212:42355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.135.254:40143
--raft_prepare_replacement_before_eviction=true
--catalog_manager_evict_excess_replicas=false with env {}
W20261115 21:02:15.731771 30837 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:15.731956 30837 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:15.731994 30837 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:15.735515 30837 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261115 21:02:15.735576 30837 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:15.735597 30837 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261115 21:02:15.735630 30837 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261115 21:02:15.739648 30837 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:42355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/wal
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.29.135.254:40143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:40143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.135.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.30837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:15.740458 30837 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:15.741163 30837 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:15.746312 30842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:15.746292 30845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:15.746285 30843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:15.746368 30837 server_base.cc:1061] running on GCE node
I20261115 21:02:15.747077 30837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:15.747794 30837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:15.748934 30837 hybrid_clock.cc:648] HybridClock initialized: now 1794776535748923 us; error 23 us; skew 500 ppm
I20261115 21:02:15.750582 30837 webserver.cc:577] Webserver started at http://127.29.135.254:37255/ using document root <none> and password file <none>
I20261115 21:02:15.751150 30837 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:15.751197 30837 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:15.751422 30837 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:15.752727 30837 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/data/instance:
uuid: "ad9d176b43744c6083075a1e19a79610"
format_stamp: "Formatted at 2026-11-15 21:02:15 on dist-test-slave-xh31"
I20261115 21:02:15.753141 30837 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/wal/instance:
uuid: "ad9d176b43744c6083075a1e19a79610"
format_stamp: "Formatted at 2026-11-15 21:02:15 on dist-test-slave-xh31"
I20261115 21:02:15.756352 30837 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20261115 21:02:15.758562 30851 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:15.759459 30837 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20261115 21:02:15.759584 30837 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/wal
uuid: "ad9d176b43744c6083075a1e19a79610"
format_stamp: "Formatted at 2026-11-15 21:02:15 on dist-test-slave-xh31"
I20261115 21:02:15.759663 30837 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:15.783799 30837 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:15.784281 30837 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:15.784417 30837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:15.790877 30837 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.254:40143
I20261115 21:02:15.790891 30903 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.254:40143 every 8 connection(s)
I20261115 21:02:15.791850 30837 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/data/info.pb
I20261115 21:02:15.794876 30904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:15.797446 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 30837
I20261115 21:02:15.797559 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/master-0/wal/instance
I20261115 21:02:15.801069 30904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610: Bootstrap starting.
I20261115 21:02:15.803740 30904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:15.804600 30904 log.cc:826] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:15.806059 30904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610: No bootstrap required, opened a new log
I20261115 21:02:15.808609 30904 raft_consensus.cc:360] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad9d176b43744c6083075a1e19a79610" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40143 } }
I20261115 21:02:15.808769 30904 raft_consensus.cc:386] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:15.808801 30904 raft_consensus.cc:748] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9d176b43744c6083075a1e19a79610, State: Initialized, Role: FOLLOWER
I20261115 21:02:15.809116 30904 consensus_queue.cc:261] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad9d176b43744c6083075a1e19a79610" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40143 } }
I20261115 21:02:15.809217 30904 raft_consensus.cc:400] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261115 21:02:15.809255 30904 raft_consensus.cc:499] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261115 21:02:15.809314 30904 raft_consensus.cc:3096] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:15.809963 30904 raft_consensus.cc:521] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad9d176b43744c6083075a1e19a79610" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40143 } }
I20261115 21:02:15.810257 30904 leader_election.cc:304] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad9d176b43744c6083075a1e19a79610; no voters:
I20261115 21:02:15.810484 30904 leader_election.cc:290] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261115 21:02:15.810616 30909 raft_consensus.cc:2826] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:15.810811 30909 raft_consensus.cc:705] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [term 1 LEADER]: Becoming Leader. State: Replica: ad9d176b43744c6083075a1e19a79610, State: Running, Role: LEADER
I20261115 21:02:15.811120 30909 consensus_queue.cc:238] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad9d176b43744c6083075a1e19a79610" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40143 } }
I20261115 21:02:15.811473 30904 sys_catalog.cc:565] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [sys.catalog]: configured and running, proceeding with master startup.
I20261115 21:02:15.812526 30911 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad9d176b43744c6083075a1e19a79610. Latest consensus state: current_term: 1 leader_uuid: "ad9d176b43744c6083075a1e19a79610" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad9d176b43744c6083075a1e19a79610" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40143 } } }
I20261115 21:02:15.812650 30911 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:15.813002 30918 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261115 21:02:15.813046 30910 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad9d176b43744c6083075a1e19a79610" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad9d176b43744c6083075a1e19a79610" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40143 } } }
I20261115 21:02:15.813225 30910 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610 [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:15.815975 30918 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261115 21:02:15.821081 30918 catalog_manager.cc:1398] Generated new cluster ID: 573d655410994c72bad6ecea30ff8f3b
I20261115 21:02:15.821184 30918 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261115 21:02:15.830644 30918 catalog_manager.cc:1421] Generated new certificate authority record
I20261115 21:02:15.831688 30918 catalog_manager.cc:1555] Loading token signing keys...
I20261115 21:02:15.843714 30918 catalog_manager.cc:6221] T 00000000000000000000000000000000 P ad9d176b43744c6083075a1e19a79610: Generated new TSK 0
I20261115 21:02:15.844300 30918 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261115 21:02:15.847889 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.193:0
--local_ip_for_outbound_sockets=127.29.135.193
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40143
--builtin_ntp_servers=127.29.135.212:42355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20261115 21:02:15.957451 30928 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:15.957613 30928 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:15.957660 30928 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:15.961000 30928 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:15.961105 30928 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.193
I20261115 21:02:15.964953 30928 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:42355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40143
--never_fsync=true
--heap_profile_path=/tmp/kudu.30928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:15.965679 30928 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:15.966369 30928 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:15.971808 30934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:15.971813 30933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:15.972035 30936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:15.972285 30928 server_base.cc:1061] running on GCE node
I20261115 21:02:15.972633 30928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:15.973187 30928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:15.974346 30928 hybrid_clock.cc:648] HybridClock initialized: now 1794776535974331 us; error 35 us; skew 500 ppm
I20261115 21:02:15.975919 30928 webserver.cc:577] Webserver started at http://127.29.135.193:39759/ using document root <none> and password file <none>
I20261115 21:02:15.976451 30928 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:15.976506 30928 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:15.976711 30928 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:15.977968 30928 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/data/instance:
uuid: "86b5077c045e4b31b6ac89e74aab1f8d"
format_stamp: "Formatted at 2026-11-15 21:02:15 on dist-test-slave-xh31"
I20261115 21:02:15.978385 30928 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/wal/instance:
uuid: "86b5077c045e4b31b6ac89e74aab1f8d"
format_stamp: "Formatted at 2026-11-15 21:02:15 on dist-test-slave-xh31"
I20261115 21:02:15.981478 30928 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20261115 21:02:15.983683 30942 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:15.984599 30928 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20261115 21:02:15.984704 30928 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/wal
uuid: "86b5077c045e4b31b6ac89e74aab1f8d"
format_stamp: "Formatted at 2026-11-15 21:02:15 on dist-test-slave-xh31"
I20261115 21:02:15.984787 30928 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:16.000949 30928 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:16.001413 30928 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:16.001569 30928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:16.002056 30928 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:16.003062 30928 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:16.003124 30928 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.003175 30928 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:16.003198 30928 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.011972 30928 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.193:34093
I20261115 21:02:16.012006 31055 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.193:34093 every 8 connection(s)
I20261115 21:02:16.012903 30928 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/data/info.pb
I20261115 21:02:16.013399 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 30928
I20261115 21:02:16.013532 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-0/wal/instance
I20261115 21:02:16.015705 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.194:0
--local_ip_for_outbound_sockets=127.29.135.194
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40143
--builtin_ntp_servers=127.29.135.212:42355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20261115 21:02:16.020625 31056 heartbeater.cc:345] Connected to a master server at 127.29.135.254:40143
I20261115 21:02:16.020823 31056 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:16.021242 31056 heartbeater.cc:508] Master 127.29.135.254:40143 requested a full tablet report, sending...
I20261115 21:02:16.022890 30868 ts_manager.cc:210] Registered new tserver with Master: 86b5077c045e4b31b6ac89e74aab1f8d (127.29.135.193:34093)
I20261115 21:02:16.024590 30868 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.193:47927
W20261115 21:02:16.117394 31059 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:16.117566 31059 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:16.117609 31059 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:16.120890 31059 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:16.120988 31059 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.194
I20261115 21:02:16.124926 31059 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:42355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40143
--never_fsync=true
--heap_profile_path=/tmp/kudu.31059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:16.125639 31059 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:16.126302 31059 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:16.131701 31064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:16.131749 31067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:16.131696 31065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:16.131893 31059 server_base.cc:1061] running on GCE node
I20261115 21:02:16.132262 31059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:16.132823 31059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:16.133965 31059 hybrid_clock.cc:648] HybridClock initialized: now 1794776536133950 us; error 31 us; skew 500 ppm
I20261115 21:02:16.135506 31059 webserver.cc:577] Webserver started at http://127.29.135.194:34677/ using document root <none> and password file <none>
I20261115 21:02:16.136031 31059 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:16.136080 31059 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:16.136274 31059 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:16.137506 31059 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/data/instance:
uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e"
format_stamp: "Formatted at 2026-11-15 21:02:16 on dist-test-slave-xh31"
I20261115 21:02:16.137907 31059 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/wal/instance:
uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e"
format_stamp: "Formatted at 2026-11-15 21:02:16 on dist-test-slave-xh31"
I20261115 21:02:16.141059 31059 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20261115 21:02:16.143198 31073 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.144054 31059 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20261115 21:02:16.144155 31059 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/wal
uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e"
format_stamp: "Formatted at 2026-11-15 21:02:16 on dist-test-slave-xh31"
I20261115 21:02:16.144232 31059 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:16.159986 31059 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:16.160434 31059 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:16.160580 31059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:16.161046 31059 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:16.161957 31059 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:16.162004 31059 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.162060 31059 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:16.162086 31059 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.171271 31059 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.194:39975
I20261115 21:02:16.171293 31186 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.194:39975 every 8 connection(s)
I20261115 21:02:16.172251 31059 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/data/info.pb
I20261115 21:02:16.179622 31187 heartbeater.cc:345] Connected to a master server at 127.29.135.254:40143
I20261115 21:02:16.179822 31187 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:16.180269 31187 heartbeater.cc:508] Master 127.29.135.254:40143 requested a full tablet report, sending...
I20261115 21:02:16.181259 30868 ts_manager.cc:210] Registered new tserver with Master: bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194:39975)
I20261115 21:02:16.181734 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 31059
I20261115 21:02:16.181833 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-1/wal/instance
I20261115 21:02:16.181821 30868 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.194:46517
I20261115 21:02:16.183806 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.195:0
--local_ip_for_outbound_sockets=127.29.135.195
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40143
--builtin_ntp_servers=127.29.135.212:42355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20261115 21:02:16.285804 31190 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:16.285981 31190 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:16.286041 31190 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:16.289445 31190 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:16.289561 31190 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.195
I20261115 21:02:16.293625 31190 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:42355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40143
--never_fsync=true
--heap_profile_path=/tmp/kudu.31190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:16.294399 31190 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:16.295113 31190 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:16.300457 31196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:16.300463 31198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:16.300463 31195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:16.300819 31190 server_base.cc:1061] running on GCE node
I20261115 21:02:16.301162 31190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:16.301683 31190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:16.302814 31190 hybrid_clock.cc:648] HybridClock initialized: now 1794776536302797 us; error 30 us; skew 500 ppm
I20261115 21:02:16.304268 31190 webserver.cc:577] Webserver started at http://127.29.135.195:45253/ using document root <none> and password file <none>
I20261115 21:02:16.304797 31190 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:16.304847 31190 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:16.305040 31190 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:16.306285 31190 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/data/instance:
uuid: "bc719b299f9840a9aa7a3e196f7f0d4e"
format_stamp: "Formatted at 2026-11-15 21:02:16 on dist-test-slave-xh31"
I20261115 21:02:16.306733 31190 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/wal/instance:
uuid: "bc719b299f9840a9aa7a3e196f7f0d4e"
format_stamp: "Formatted at 2026-11-15 21:02:16 on dist-test-slave-xh31"
I20261115 21:02:16.310040 31190 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20261115 21:02:16.312321 31204 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.313261 31190 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20261115 21:02:16.313393 31190 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/wal
uuid: "bc719b299f9840a9aa7a3e196f7f0d4e"
format_stamp: "Formatted at 2026-11-15 21:02:16 on dist-test-slave-xh31"
I20261115 21:02:16.313498 31190 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:16.334313 31190 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:16.334874 31190 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:16.335008 31190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:16.335495 31190 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:16.336452 31190 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:16.336498 31190 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.336539 31190 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:16.336560 31190 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.345858 31190 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.195:37609
I20261115 21:02:16.345904 31317 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.195:37609 every 8 connection(s)
I20261115 21:02:16.346909 31190 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/data/info.pb
I20261115 21:02:16.349273 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 31190
I20261115 21:02:16.349367 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-2/wal/instance
I20261115 21:02:16.351255 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.196:0
--local_ip_for_outbound_sockets=127.29.135.196
--webserver_interface=127.29.135.196
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40143
--builtin_ntp_servers=127.29.135.212:42355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20261115 21:02:16.354389 31318 heartbeater.cc:345] Connected to a master server at 127.29.135.254:40143
I20261115 21:02:16.354627 31318 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:16.355080 31318 heartbeater.cc:508] Master 127.29.135.254:40143 requested a full tablet report, sending...
I20261115 21:02:16.356101 30868 ts_manager.cc:210] Registered new tserver with Master: bc719b299f9840a9aa7a3e196f7f0d4e (127.29.135.195:37609)
I20261115 21:02:16.356818 30868 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.195:39947
W20261115 21:02:16.453699 31321 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:16.453850 31321 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:16.453891 31321 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:16.457268 31321 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:16.457374 31321 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.196
I20261115 21:02:16.461185 31321 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:42355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.29.135.196
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40143
--never_fsync=true
--heap_profile_path=/tmp/kudu.31321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.196
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:16.461925 31321 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:16.462669 31321 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:16.468111 31326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:16.468397 31329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:16.468428 31327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:16.468479 31321 server_base.cc:1061] running on GCE node
I20261115 21:02:16.468832 31321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:16.469338 31321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:16.470530 31321 hybrid_clock.cc:648] HybridClock initialized: now 1794776536470477 us; error 27 us; skew 500 ppm
I20261115 21:02:16.471966 31321 webserver.cc:577] Webserver started at http://127.29.135.196:35467/ using document root <none> and password file <none>
I20261115 21:02:16.472528 31321 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:16.472574 31321 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:16.472761 31321 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:16.473994 31321 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/data/instance:
uuid: "580cf0b3a06347a992f29f3f18afd1c4"
format_stamp: "Formatted at 2026-11-15 21:02:16 on dist-test-slave-xh31"
I20261115 21:02:16.474392 31321 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/wal/instance:
uuid: "580cf0b3a06347a992f29f3f18afd1c4"
format_stamp: "Formatted at 2026-11-15 21:02:16 on dist-test-slave-xh31"
I20261115 21:02:16.477810 31321 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20261115 21:02:16.480212 31335 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.481189 31321 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20261115 21:02:16.481290 31321 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/wal
uuid: "580cf0b3a06347a992f29f3f18afd1c4"
format_stamp: "Formatted at 2026-11-15 21:02:16 on dist-test-slave-xh31"
I20261115 21:02:16.481372 31321 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:16.496408 31321 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:16.496919 31321 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:16.497076 31321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:16.497597 31321 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:16.498623 31321 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:16.498685 31321 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.498737 31321 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:16.498765 31321 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:16.507241 31321 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.196:44797
I20261115 21:02:16.507277 31448 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.196:44797 every 8 connection(s)
I20261115 21:02:16.508172 31321 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/data/info.pb
I20261115 21:02:16.515327 31449 heartbeater.cc:345] Connected to a master server at 127.29.135.254:40143
I20261115 21:02:16.515538 31449 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:16.515965 31449 heartbeater.cc:508] Master 127.29.135.254:40143 requested a full tablet report, sending...
I20261115 21:02:16.516942 30868 ts_manager.cc:210] Registered new tserver with Master: 580cf0b3a06347a992f29f3f18afd1c4 (127.29.135.196:44797)
I20261115 21:02:16.517032 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 31321
I20261115 21:02:16.517140 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794776526014215-30239-0/minicluster-data/ts-3/wal/instance
I20261115 21:02:16.517639 30868 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.196:37587
I20261115 21:02:16.519377 30239 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261115 21:02:16.534588 30868 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40130:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261115 21:02:16.551156 30990 tablet_service.cc:1500] Processing CreateTablet for tablet 0c19f3ab161848a28a1a55db6a4e3ba2 (DEFAULT_TABLE table=test-workload [id=7a6e1fe2a18044cfb119ed70f89ac5a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:16.551159 31252 tablet_service.cc:1500] Processing CreateTablet for tablet 0c19f3ab161848a28a1a55db6a4e3ba2 (DEFAULT_TABLE table=test-workload [id=7a6e1fe2a18044cfb119ed70f89ac5a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:16.551159 31383 tablet_service.cc:1500] Processing CreateTablet for tablet 0c19f3ab161848a28a1a55db6a4e3ba2 (DEFAULT_TABLE table=test-workload [id=7a6e1fe2a18044cfb119ed70f89ac5a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:16.552042 30990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c19f3ab161848a28a1a55db6a4e3ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:16.552063 31252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c19f3ab161848a28a1a55db6a4e3ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:16.552073 31383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c19f3ab161848a28a1a55db6a4e3ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:16.556990 31471 tablet_bootstrap.cc:492] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d: Bootstrap starting.
I20261115 21:02:16.558394 31472 tablet_bootstrap.cc:492] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e: Bootstrap starting.
I20261115 21:02:16.560554 31471 tablet_bootstrap.cc:654] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:16.560554 31472 tablet_bootstrap.cc:654] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:16.561450 31471 log.cc:826] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:16.561450 31472 log.cc:826] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:16.562052 31473 tablet_bootstrap.cc:492] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4: Bootstrap starting.
I20261115 21:02:16.562937 31472 tablet_bootstrap.cc:492] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e: No bootstrap required, opened a new log
I20261115 21:02:16.563127 31472 ts_tablet_manager.cc:1449] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20261115 21:02:16.563184 31471 tablet_bootstrap.cc:492] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d: No bootstrap required, opened a new log
I20261115 21:02:16.563373 31471 ts_tablet_manager.cc:1449] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20261115 21:02:16.564286 31473 tablet_bootstrap.cc:654] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:16.565114 31473 log.cc:826] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:16.565851 31472 raft_consensus.cc:360] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } }
I20261115 21:02:16.566013 31472 raft_consensus.cc:386] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:16.566082 31472 raft_consensus.cc:748] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc719b299f9840a9aa7a3e196f7f0d4e, State: Initialized, Role: FOLLOWER
I20261115 21:02:16.566556 31473 tablet_bootstrap.cc:492] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4: No bootstrap required, opened a new log
I20261115 21:02:16.566476 31472 consensus_queue.cc:261] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } }
I20261115 21:02:16.566465 31471 raft_consensus.cc:360] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } }
I20261115 21:02:16.566664 31471 raft_consensus.cc:386] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:16.566744 31471 raft_consensus.cc:748] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b5077c045e4b31b6ac89e74aab1f8d, State: Initialized, Role: FOLLOWER
I20261115 21:02:16.566753 31473 ts_tablet_manager.cc:1449] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20261115 21:02:16.567109 31318 heartbeater.cc:500] Master 127.29.135.254:40143 was elected leader, sending a full tablet report...
I20261115 21:02:16.567209 31472 ts_tablet_manager.cc:1480] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20261115 21:02:16.568370 31471 consensus_queue.cc:261] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } }
I20261115 21:02:16.569120 31471 ts_tablet_manager.cc:1480] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20261115 21:02:16.569463 31056 heartbeater.cc:500] Master 127.29.135.254:40143 was elected leader, sending a full tablet report...
I20261115 21:02:16.569581 31473 raft_consensus.cc:360] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } }
I20261115 21:02:16.569715 31473 raft_consensus.cc:386] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:16.569748 31473 raft_consensus.cc:748] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580cf0b3a06347a992f29f3f18afd1c4, State: Initialized, Role: FOLLOWER
I20261115 21:02:16.570071 31473 consensus_queue.cc:261] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } }
I20261115 21:02:16.570850 31449 heartbeater.cc:500] Master 127.29.135.254:40143 was elected leader, sending a full tablet report...
I20261115 21:02:16.571748 31473 ts_tablet_manager.cc:1480] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20261115 21:02:16.597640 31319 tablet.cc:2441] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261115 21:02:16.758913 31450 tablet.cc:2441] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261115 21:02:16.762953 31477 raft_consensus.cc:499] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261115 21:02:16.763121 31477 raft_consensus.cc:521] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } }
W20261115 21:02:16.764079 31057 tablet.cc:2441] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261115 21:02:16.764277 31477 leader_election.cc:290] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86b5077c045e4b31b6ac89e74aab1f8d (127.29.135.193:34093), 580cf0b3a06347a992f29f3f18afd1c4 (127.29.135.196:44797)
I20261115 21:02:16.767971 31010 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0c19f3ab161848a28a1a55db6a4e3ba2" candidate_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" is_pre_election: true
I20261115 21:02:16.768000 31403 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0c19f3ab161848a28a1a55db6a4e3ba2" candidate_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580cf0b3a06347a992f29f3f18afd1c4" is_pre_election: true
I20261115 21:02:16.768204 31403 raft_consensus.cc:2490] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc719b299f9840a9aa7a3e196f7f0d4e in term 0.
I20261115 21:02:16.768198 31010 raft_consensus.cc:2490] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc719b299f9840a9aa7a3e196f7f0d4e in term 0.
I20261115 21:02:16.768587 31207 leader_election.cc:304] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580cf0b3a06347a992f29f3f18afd1c4, bc719b299f9840a9aa7a3e196f7f0d4e; no voters:
I20261115 21:02:16.768784 31477 raft_consensus.cc:2826] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261115 21:02:16.768848 31477 raft_consensus.cc:499] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261115 21:02:16.768898 31477 raft_consensus.cc:3096] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:16.769714 31477 raft_consensus.cc:521] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } }
I20261115 21:02:16.769999 31477 leader_election.cc:290] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [CANDIDATE]: Term 1 election: Requested vote from peers 86b5077c045e4b31b6ac89e74aab1f8d (127.29.135.193:34093), 580cf0b3a06347a992f29f3f18afd1c4 (127.29.135.196:44797)
I20261115 21:02:16.770306 31403 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0c19f3ab161848a28a1a55db6a4e3ba2" candidate_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580cf0b3a06347a992f29f3f18afd1c4"
I20261115 21:02:16.770296 31010 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0c19f3ab161848a28a1a55db6a4e3ba2" candidate_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b5077c045e4b31b6ac89e74aab1f8d"
I20261115 21:02:16.770404 31403 raft_consensus.cc:3096] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:16.770411 31010 raft_consensus.cc:3096] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:16.771199 31403 raft_consensus.cc:2490] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc719b299f9840a9aa7a3e196f7f0d4e in term 1.
I20261115 21:02:16.771436 31207 leader_election.cc:304] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580cf0b3a06347a992f29f3f18afd1c4, bc719b299f9840a9aa7a3e196f7f0d4e; no voters:
I20261115 21:02:16.771627 31477 raft_consensus.cc:2826] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:16.771678 31010 raft_consensus.cc:2490] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc719b299f9840a9aa7a3e196f7f0d4e in term 1.
I20261115 21:02:16.771804 31477 raft_consensus.cc:705] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 1 LEADER]: Becoming Leader. State: Replica: bc719b299f9840a9aa7a3e196f7f0d4e, State: Running, Role: LEADER
I20261115 21:02:16.772127 31477 consensus_queue.cc:238] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } }
I20261115 21:02:16.775266 31477 consensus_queue.cc:1049] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:16.775406 31481 consensus_queue.cc:1049] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:16.775292 30866 catalog_manager.cc:5840] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bc719b299f9840a9aa7a3e196f7f0d4e (127.29.135.195). New cstate: current_term: 1 leader_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20261115 21:02:16.848345 31272 consensus_queue.cc:238] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: true } }
I20261115 21:02:16.849615 31010 raft_consensus.cc:1285] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [term 1 FOLLOWER]: Refusing update from remote peer bc719b299f9840a9aa7a3e196f7f0d4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20261115 21:02:16.849860 31403 raft_consensus.cc:1285] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [term 1 FOLLOWER]: Refusing update from remote peer bc719b299f9840a9aa7a3e196f7f0d4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20261115 21:02:16.850265 31483 consensus_queue.cc:1049] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20261115 21:02:16.850420 31477 consensus_queue.cc:1049] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20261115 21:02:16.852228 31477 raft_consensus.cc:2983] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: true } } }
I20261115 21:02:16.852613 31403 raft_consensus.cc:2983] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: true } } }
W20261115 21:02:16.853317 31206 consensus_peers.cc:597] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e -> Peer bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194:39975): Couldn't send request to peer bec9145a87fc46dcb3f1d7ee9b05da2e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c19f3ab161848a28a1a55db6a4e3ba2. This is attempt 1: this message will repeat every 5th retry.
I20261115 21:02:16.853264 31010 raft_consensus.cc:2983] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: true } } }
I20261115 21:02:16.855051 30867 catalog_manager.cc:5840] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e reported cstate change: config changed from index -1 to 2, NON_VOTER bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194) added. New cstate: current_term: 1 leader_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20261115 21:02:17.183902 31187 heartbeater.cc:500] Master 127.29.135.254:40143 was elected leader, sending a full tablet report...
I20261115 21:02:17.367671 31492 ts_tablet_manager.cc:979] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: Initiating tablet copy from peer bc719b299f9840a9aa7a3e196f7f0d4e (127.29.135.195:37609)
I20261115 21:02:17.368129 31492 tablet_copy_client.cc:323] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: tablet copy: Beginning tablet copy session from remote peer at address 127.29.135.195:37609
I20261115 21:02:17.371460 31292 tablet_copy_service.cc:140] P bc719b299f9840a9aa7a3e196f7f0d4e: Received BeginTabletCopySession request for tablet 0c19f3ab161848a28a1a55db6a4e3ba2 from peer bec9145a87fc46dcb3f1d7ee9b05da2e ({username='slave'} at 127.29.135.194:58653)
I20261115 21:02:17.371572 31292 tablet_copy_service.cc:161] P bc719b299f9840a9aa7a3e196f7f0d4e: Beginning new tablet copy session on tablet 0c19f3ab161848a28a1a55db6a4e3ba2 from peer bec9145a87fc46dcb3f1d7ee9b05da2e at {username='slave'} at 127.29.135.194:58653: session id = bec9145a87fc46dcb3f1d7ee9b05da2e-0c19f3ab161848a28a1a55db6a4e3ba2
I20261115 21:02:17.372700 31292 tablet_copy_source_session.cc:215] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e: Tablet Copy: opened 0 blocks and 1 log segments
I20261115 21:02:17.375907 31492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c19f3ab161848a28a1a55db6a4e3ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:17.380177 31492 tablet_copy_client.cc:806] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: tablet copy: Starting download of 0 data blocks...
I20261115 21:02:17.380332 31492 tablet_copy_client.cc:670] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: tablet copy: Starting download of 1 WAL segments...
I20261115 21:02:17.381734 31492 tablet_copy_client.cc:538] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261115 21:02:17.383083 31492 tablet_bootstrap.cc:492] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: Bootstrap starting.
I20261115 21:02:17.386891 31492 log.cc:826] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:17.388866 31492 tablet_bootstrap.cc:492] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261115 21:02:17.389091 31492 tablet_bootstrap.cc:492] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: Bootstrap complete.
I20261115 21:02:17.389323 31492 ts_tablet_manager.cc:1449] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20261115 21:02:17.391425 31492 raft_consensus.cc:360] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: true } }
I20261115 21:02:17.391597 31492 raft_consensus.cc:748] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bec9145a87fc46dcb3f1d7ee9b05da2e, State: Initialized, Role: LEARNER
I20261115 21:02:17.391866 31492 consensus_queue.cc:261] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: true } }
I20261115 21:02:17.392475 31492 ts_tablet_manager.cc:1480] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20261115 21:02:17.393014 31292 tablet_copy_service.cc:342] P bc719b299f9840a9aa7a3e196f7f0d4e: Request end of tablet copy session bec9145a87fc46dcb3f1d7ee9b05da2e-0c19f3ab161848a28a1a55db6a4e3ba2 received from {username='slave'} at 127.29.135.194:58653
I20261115 21:02:17.393097 31292 tablet_copy_service.cc:434] P bc719b299f9840a9aa7a3e196f7f0d4e: ending tablet copy session bec9145a87fc46dcb3f1d7ee9b05da2e-0c19f3ab161848a28a1a55db6a4e3ba2 on tablet 0c19f3ab161848a28a1a55db6a4e3ba2 with peer bec9145a87fc46dcb3f1d7ee9b05da2e
I20261115 21:02:17.792224 31141 raft_consensus.cc:1227] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20261115 21:02:18.391873 31500 raft_consensus.cc:1074] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e: attempting to promote NON_VOTER bec9145a87fc46dcb3f1d7ee9b05da2e to VOTER
I20261115 21:02:18.392326 31500 consensus_queue.cc:238] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: false } }
I20261115 21:02:18.393599 31141 raft_consensus.cc:1285] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e [term 1 LEARNER]: Refusing update from remote peer bc719b299f9840a9aa7a3e196f7f0d4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20261115 21:02:18.393630 31010 raft_consensus.cc:1285] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [term 1 FOLLOWER]: Refusing update from remote peer bc719b299f9840a9aa7a3e196f7f0d4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20261115 21:02:18.393646 31403 raft_consensus.cc:1285] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [term 1 FOLLOWER]: Refusing update from remote peer bc719b299f9840a9aa7a3e196f7f0d4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20261115 21:02:18.393996 31499 consensus_queue.cc:1049] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20261115 21:02:18.394167 31503 consensus_queue.cc:1049] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20261115 21:02:18.394315 31500 consensus_queue.cc:1049] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20261115 21:02:18.396101 31503 raft_consensus.cc:2983] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: false } } }
I20261115 21:02:18.396462 31010 raft_consensus.cc:2983] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 86b5077c045e4b31b6ac89e74aab1f8d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: false } } }
I20261115 21:02:18.397598 31141 raft_consensus.cc:2983] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bec9145a87fc46dcb3f1d7ee9b05da2e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: false } } }
I20261115 21:02:18.398686 31403 raft_consensus.cc:2983] T 0c19f3ab161848a28a1a55db6a4e3ba2 P 580cf0b3a06347a992f29f3f18afd1c4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: false } } }
I20261115 21:02:18.398833 30868 catalog_manager.cc:5840] T 0c19f3ab161848a28a1a55db6a4e3ba2 P bc719b299f9840a9aa7a3e196f7f0d4e reported cstate change: config changed from index 2 to 3, bec9145a87fc46dcb3f1d7ee9b05da2e (127.29.135.194) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bc719b299f9840a9aa7a3e196f7f0d4e" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86b5077c045e4b31b6ac89e74aab1f8d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 34093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "580cf0b3a06347a992f29f3f18afd1c4" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 44797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bec9145a87fc46dcb3f1d7ee9b05da2e" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39975 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20261115 21:02:18.409457 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 30928
I20261115 21:02:18.414937 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 31059
I20261115 21:02:18.419950 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 31190
I20261115 21:02:18.425025 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 31321
I20261115 21:02:18.429961 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 30837
2026-11-15T21:02:18Z chronyd exiting
[ OK ] RaftConfigChangeITest.TestNonVoterPromotion (2824 ms)
[ RUN ] RaftConfigChangeITest.TestBulkChangeConfig
2026-11-15T21:02:18Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-15T21:02:18Z Disabled control of system clock
I20261115 21:02:18.456713 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:43903
--webserver_interface=127.29.135.254
--webserver_port=0
--builtin_ntp_servers=127.29.135.212:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.135.254:43903
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261115 21:02:18.552500 31518 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:18.552660 31518 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:18.552703 31518 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:18.556094 31518 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261115 21:02:18.556147 31518 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:18.556166 31518 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261115 21:02:18.556183 31518 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261115 21:02:18.560084 31518 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.29.135.254:43903
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:43903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.135.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.31518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:18.560847 31518 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:18.561558 31518 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:18.566879 31526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:18.566951 31523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:18.566907 31524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:18.567342 31518 server_base.cc:1061] running on GCE node
I20261115 21:02:18.567739 31518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:18.568419 31518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:18.569595 31518 hybrid_clock.cc:648] HybridClock initialized: now 1794776538569554 us; error 56 us; skew 500 ppm
I20261115 21:02:18.573835 31518 webserver.cc:577] Webserver started at http://127.29.135.254:35925/ using document root <none> and password file <none>
I20261115 21:02:18.574522 31518 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:18.574589 31518 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:18.574853 31518 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:18.576303 31518 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/data/instance:
uuid: "3dc987c8ce884f0286eb66c90fc43265"
format_stamp: "Formatted at 2026-11-15 21:02:18 on dist-test-slave-xh31"
I20261115 21:02:18.576762 31518 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/wal/instance:
uuid: "3dc987c8ce884f0286eb66c90fc43265"
format_stamp: "Formatted at 2026-11-15 21:02:18 on dist-test-slave-xh31"
I20261115 21:02:18.580235 31518 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20261115 21:02:18.582690 31532 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:18.583684 31518 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:18.583818 31518 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/wal
uuid: "3dc987c8ce884f0286eb66c90fc43265"
format_stamp: "Formatted at 2026-11-15 21:02:18 on dist-test-slave-xh31"
I20261115 21:02:18.583894 31518 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:18.600726 31518 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:18.601212 31518 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:18.601330 31518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:18.607888 31584 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.254:43903 every 8 connection(s)
I20261115 21:02:18.607892 31518 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.254:43903
I20261115 21:02:18.608911 31518 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/data/info.pb
I20261115 21:02:18.612067 31585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:18.613348 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 31518
I20261115 21:02:18.613452 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/master-0/wal/instance
I20261115 21:02:18.617307 31585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265: Bootstrap starting.
I20261115 21:02:18.619720 31585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:18.620465 31585 log.cc:826] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:18.621770 31585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265: No bootstrap required, opened a new log
I20261115 21:02:18.624408 31585 raft_consensus.cc:360] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc987c8ce884f0286eb66c90fc43265" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 43903 } }
I20261115 21:02:18.624614 31585 raft_consensus.cc:386] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:18.624692 31585 raft_consensus.cc:748] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dc987c8ce884f0286eb66c90fc43265, State: Initialized, Role: FOLLOWER
I20261115 21:02:18.625128 31585 consensus_queue.cc:261] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc987c8ce884f0286eb66c90fc43265" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 43903 } }
I20261115 21:02:18.625547 31585 raft_consensus.cc:400] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261115 21:02:18.625651 31585 raft_consensus.cc:499] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261115 21:02:18.625775 31585 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:18.626947 31585 raft_consensus.cc:521] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc987c8ce884f0286eb66c90fc43265" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 43903 } }
I20261115 21:02:18.627328 31585 leader_election.cc:304] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3dc987c8ce884f0286eb66c90fc43265; no voters:
I20261115 21:02:18.627599 31585 leader_election.cc:290] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261115 21:02:18.627698 31590 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:18.627879 31590 raft_consensus.cc:705] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [term 1 LEADER]: Becoming Leader. State: Replica: 3dc987c8ce884f0286eb66c90fc43265, State: Running, Role: LEADER
I20261115 21:02:18.628216 31590 consensus_queue.cc:238] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc987c8ce884f0286eb66c90fc43265" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 43903 } }
I20261115 21:02:18.628651 31585 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [sys.catalog]: configured and running, proceeding with master startup.
I20261115 21:02:18.629704 31592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3dc987c8ce884f0286eb66c90fc43265. Latest consensus state: current_term: 1 leader_uuid: "3dc987c8ce884f0286eb66c90fc43265" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc987c8ce884f0286eb66c90fc43265" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 43903 } } }
I20261115 21:02:18.629827 31592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:18.630182 31599 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261115 21:02:18.630282 31591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3dc987c8ce884f0286eb66c90fc43265" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc987c8ce884f0286eb66c90fc43265" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 43903 } } }
I20261115 21:02:18.630435 31591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265 [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:18.633513 31599 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261115 21:02:18.639024 31599 catalog_manager.cc:1398] Generated new cluster ID: 32acff76b15647d897c6df5c13599415
I20261115 21:02:18.639089 31599 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261115 21:02:18.653745 31599 catalog_manager.cc:1421] Generated new certificate authority record
I20261115 21:02:18.654662 31599 catalog_manager.cc:1555] Loading token signing keys...
I20261115 21:02:18.661263 31599 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 3dc987c8ce884f0286eb66c90fc43265: Generated new TSK 0
I20261115 21:02:18.661803 31599 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261115 21:02:18.670863 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.193:0
--local_ip_for_outbound_sockets=127.29.135.193
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:43903
--builtin_ntp_servers=127.29.135.212:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261115 21:02:18.791594 31609 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261115 21:02:18.791811 31609 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:18.791857 31609 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:18.791925 31609 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:18.795639 31609 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:18.795787 31609 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.193
I20261115 21:02:18.800028 31609 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:43903
--never_fsync=true
--heap_profile_path=/tmp/kudu.31609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:18.800879 31609 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:18.801620 31609 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:18.807981 31617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:18.808090 31614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:18.807994 31615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:18.809201 31609 server_base.cc:1061] running on GCE node
I20261115 21:02:18.809715 31609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:18.810317 31609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:18.811465 31609 hybrid_clock.cc:648] HybridClock initialized: now 1794776538811431 us; error 56 us; skew 500 ppm
I20261115 21:02:18.813189 31609 webserver.cc:577] Webserver started at http://127.29.135.193:36619/ using document root <none> and password file <none>
I20261115 21:02:18.813797 31609 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:18.813851 31609 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:18.814072 31609 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:18.815457 31609 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/data/instance:
uuid: "7403415722954f7d8621d306bb174044"
format_stamp: "Formatted at 2026-11-15 21:02:18 on dist-test-slave-xh31"
I20261115 21:02:18.815899 31609 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/wal/instance:
uuid: "7403415722954f7d8621d306bb174044"
format_stamp: "Formatted at 2026-11-15 21:02:18 on dist-test-slave-xh31"
I20261115 21:02:18.819264 31609 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20261115 21:02:18.821504 31623 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:18.822588 31609 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20261115 21:02:18.822707 31609 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/wal
uuid: "7403415722954f7d8621d306bb174044"
format_stamp: "Formatted at 2026-11-15 21:02:18 on dist-test-slave-xh31"
I20261115 21:02:18.822806 31609 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:18.839267 31609 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:18.839823 31609 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:18.839977 31609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:18.840507 31609 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:18.841538 31609 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:18.841588 31609 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:18.841647 31609 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:18.841674 31609 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:18.850037 31609 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.193:40609
I20261115 21:02:18.850080 31736 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.193:40609 every 8 connection(s)
I20261115 21:02:18.851035 31609 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/data/info.pb
I20261115 21:02:18.858399 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 31609
I20261115 21:02:18.858564 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-0/wal/instance
I20261115 21:02:18.858774 31737 heartbeater.cc:345] Connected to a master server at 127.29.135.254:43903
I20261115 21:02:18.859014 31737 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:18.859512 31737 heartbeater.cc:508] Master 127.29.135.254:43903 requested a full tablet report, sending...
I20261115 21:02:18.860710 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.194:0
--local_ip_for_outbound_sockets=127.29.135.194
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:43903
--builtin_ntp_servers=127.29.135.212:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
I20261115 21:02:18.861104 31549 ts_manager.cc:210] Registered new tserver with Master: 7403415722954f7d8621d306bb174044 (127.29.135.193:40609)
I20261115 21:02:18.862680 31549 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.193:38995
W20261115 21:02:18.961567 31740 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261115 21:02:18.961767 31740 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:18.961799 31740 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:18.961860 31740 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:18.965329 31740 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:18.965477 31740 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.194
I20261115 21:02:18.969560 31740 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:43903
--never_fsync=true
--heap_profile_path=/tmp/kudu.31740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:18.970384 31740 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:18.971127 31740 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:18.976884 31746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:18.976951 31740 server_base.cc:1061] running on GCE node
W20261115 21:02:18.976884 31745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:18.977144 31748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:18.977473 31740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:18.978001 31740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:18.979149 31740 hybrid_clock.cc:648] HybridClock initialized: now 1794776538979127 us; error 36 us; skew 500 ppm
I20261115 21:02:18.980661 31740 webserver.cc:577] Webserver started at http://127.29.135.194:37709/ using document root <none> and password file <none>
I20261115 21:02:18.981204 31740 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:18.981253 31740 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:18.981462 31740 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:18.982761 31740 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/data/instance:
uuid: "715f35676eb84845b460e696c9c7d27a"
format_stamp: "Formatted at 2026-11-15 21:02:18 on dist-test-slave-xh31"
I20261115 21:02:18.983166 31740 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/wal/instance:
uuid: "715f35676eb84845b460e696c9c7d27a"
format_stamp: "Formatted at 2026-11-15 21:02:18 on dist-test-slave-xh31"
I20261115 21:02:18.986171 31740 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20261115 21:02:18.988489 31754 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:18.989398 31740 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:18.989509 31740 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/wal
uuid: "715f35676eb84845b460e696c9c7d27a"
format_stamp: "Formatted at 2026-11-15 21:02:18 on dist-test-slave-xh31"
I20261115 21:02:18.989585 31740 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:19.024598 31740 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:19.025136 31740 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:19.025290 31740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:19.025804 31740 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:19.026854 31740 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:19.026925 31740 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:19.026995 31740 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:19.027019 31740 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:19.035732 31740 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.194:38899
I20261115 21:02:19.035763 31867 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.194:38899 every 8 connection(s)
I20261115 21:02:19.036732 31740 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/data/info.pb
I20261115 21:02:19.037213 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 31740
I20261115 21:02:19.037326 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-1/wal/instance
I20261115 21:02:19.039568 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.195:0
--local_ip_for_outbound_sockets=127.29.135.195
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:43903
--builtin_ntp_servers=127.29.135.212:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
I20261115 21:02:19.044394 31868 heartbeater.cc:345] Connected to a master server at 127.29.135.254:43903
I20261115 21:02:19.044615 31868 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:19.045092 31868 heartbeater.cc:508] Master 127.29.135.254:43903 requested a full tablet report, sending...
I20261115 21:02:19.046123 31549 ts_manager.cc:210] Registered new tserver with Master: 715f35676eb84845b460e696c9c7d27a (127.29.135.194:38899)
I20261115 21:02:19.046703 31549 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.194:42361
W20261115 21:02:19.155545 31871 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261115 21:02:19.155757 31871 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:19.155792 31871 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:19.155862 31871 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:19.159576 31871 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:19.159736 31871 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.195
I20261115 21:02:19.164521 31871 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:43903
--never_fsync=true
--heap_profile_path=/tmp/kudu.31871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:19.165406 31871 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:19.166142 31871 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:19.173225 31879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:19.173218 31876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:19.173223 31877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:19.173696 31871 server_base.cc:1061] running on GCE node
I20261115 21:02:19.174057 31871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:19.174659 31871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:19.175815 31871 hybrid_clock.cc:648] HybridClock initialized: now 1794776539175803 us; error 27 us; skew 500 ppm
I20261115 21:02:19.177594 31871 webserver.cc:577] Webserver started at http://127.29.135.195:42721/ using document root <none> and password file <none>
I20261115 21:02:19.178212 31871 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:19.178267 31871 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:19.178464 31871 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:19.179847 31871 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/data/instance:
uuid: "b367994359394faea71a66950795aa50"
format_stamp: "Formatted at 2026-11-15 21:02:19 on dist-test-slave-xh31"
I20261115 21:02:19.180428 31871 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/wal/instance:
uuid: "b367994359394faea71a66950795aa50"
format_stamp: "Formatted at 2026-11-15 21:02:19 on dist-test-slave-xh31"
I20261115 21:02:19.184002 31871 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20261115 21:02:19.186785 31885 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:19.188414 31871 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20261115 21:02:19.190433 31871 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/wal
uuid: "b367994359394faea71a66950795aa50"
format_stamp: "Formatted at 2026-11-15 21:02:19 on dist-test-slave-xh31"
I20261115 21:02:19.190886 31871 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:19.203689 31871 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:19.204365 31871 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:19.204627 31871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:19.205406 31871 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:19.206835 31871 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:19.206892 31871 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:19.206940 31871 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:19.206954 31871 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:19.219211 31871 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.195:45161
I20261115 21:02:19.219237 31998 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.195:45161 every 8 connection(s)
I20261115 21:02:19.220273 31871 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/data/info.pb
I20261115 21:02:19.226171 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 31871
I20261115 21:02:19.226321 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-2/wal/instance
I20261115 21:02:19.229928 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.196:0
--local_ip_for_outbound_sockets=127.29.135.196
--webserver_interface=127.29.135.196
--webserver_port=0
--tserver_master_addrs=127.29.135.254:43903
--builtin_ntp_servers=127.29.135.212:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
I20261115 21:02:19.230185 31999 heartbeater.cc:345] Connected to a master server at 127.29.135.254:43903
I20261115 21:02:19.230581 31999 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:19.231438 31999 heartbeater.cc:508] Master 127.29.135.254:43903 requested a full tablet report, sending...
I20261115 21:02:19.233140 31549 ts_manager.cc:210] Registered new tserver with Master: b367994359394faea71a66950795aa50 (127.29.135.195:45161)
I20261115 21:02:19.233974 31549 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.195:53873
W20261115 21:02:19.355218 32002 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261115 21:02:19.355427 32002 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:19.355448 32002 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:19.355490 32002 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:19.359166 32002 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:19.359278 32002 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.196
I20261115 21:02:19.363250 32002 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.29.135.196
--webserver_port=0
--tserver_master_addrs=127.29.135.254:43903
--never_fsync=true
--heap_profile_path=/tmp/kudu.32002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.196
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:19.364058 32002 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:19.364774 32002 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:19.370630 32007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:19.370632 32008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:19.370759 32002 server_base.cc:1061] running on GCE node
W20261115 21:02:19.370662 32010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:19.371209 32002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:19.371737 32002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:19.372884 32002 hybrid_clock.cc:648] HybridClock initialized: now 1794776539372859 us; error 39 us; skew 500 ppm
I20261115 21:02:19.374444 32002 webserver.cc:577] Webserver started at http://127.29.135.196:42439/ using document root <none> and password file <none>
I20261115 21:02:19.375062 32002 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:19.375114 32002 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:19.375362 32002 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:19.377115 32002 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/data/instance:
uuid: "a8ae8c02d6e34647bca4953f59b3cf71"
format_stamp: "Formatted at 2026-11-15 21:02:19 on dist-test-slave-xh31"
I20261115 21:02:19.377635 32002 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/wal/instance:
uuid: "a8ae8c02d6e34647bca4953f59b3cf71"
format_stamp: "Formatted at 2026-11-15 21:02:19 on dist-test-slave-xh31"
I20261115 21:02:19.381927 32002 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20261115 21:02:19.384899 32016 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:19.386044 32002 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:19.386160 32002 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/wal
uuid: "a8ae8c02d6e34647bca4953f59b3cf71"
format_stamp: "Formatted at 2026-11-15 21:02:19 on dist-test-slave-xh31"
I20261115 21:02:19.386260 32002 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:19.410686 32002 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:19.411188 32002 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:19.411316 32002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:19.411777 32002 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:19.412711 32002 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:19.412757 32002 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:19.412798 32002 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:19.412818 32002 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:19.421144 32002 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.196:34143
I20261115 21:02:19.421182 32129 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.196:34143 every 8 connection(s)
I20261115 21:02:19.422128 32002 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/data/info.pb
I20261115 21:02:19.427255 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 32002
I20261115 21:02:19.427371 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794776526014215-30239-0/minicluster-data/ts-3/wal/instance
I20261115 21:02:19.429886 32130 heartbeater.cc:345] Connected to a master server at 127.29.135.254:43903
I20261115 21:02:19.430107 32130 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:19.430610 32130 heartbeater.cc:508] Master 127.29.135.254:43903 requested a full tablet report, sending...
I20261115 21:02:19.431565 31548 ts_manager.cc:210] Registered new tserver with Master: a8ae8c02d6e34647bca4953f59b3cf71 (127.29.135.196:34143)
I20261115 21:02:19.432112 31548 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.196:35457
I20261115 21:02:19.441144 30239 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261115 21:02:19.456550 31548 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36056:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261115 21:02:19.472287 32064 tablet_service.cc:1500] Processing CreateTablet for tablet cfe14815e02e4d6eab658ba9a0b98154 (DEFAULT_TABLE table=test-workload [id=e6da95effa034e34bf613a06131c7d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:19.472388 31933 tablet_service.cc:1500] Processing CreateTablet for tablet cfe14815e02e4d6eab658ba9a0b98154 (DEFAULT_TABLE table=test-workload [id=e6da95effa034e34bf613a06131c7d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:19.472388 31802 tablet_service.cc:1500] Processing CreateTablet for tablet cfe14815e02e4d6eab658ba9a0b98154 (DEFAULT_TABLE table=test-workload [id=e6da95effa034e34bf613a06131c7d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:19.473150 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfe14815e02e4d6eab658ba9a0b98154. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:19.473248 31802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfe14815e02e4d6eab658ba9a0b98154. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:19.473248 31933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfe14815e02e4d6eab658ba9a0b98154. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:19.477993 32152 tablet_bootstrap.cc:492] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71: Bootstrap starting.
I20261115 21:02:19.479367 32154 tablet_bootstrap.cc:492] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a: Bootstrap starting.
I20261115 21:02:19.480286 32152 tablet_bootstrap.cc:654] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:19.480981 32152 log.cc:826] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:19.481105 32154 tablet_bootstrap.cc:654] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:19.481534 32153 tablet_bootstrap.cc:492] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50: Bootstrap starting.
I20261115 21:02:19.481899 32154 log.cc:826] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:19.482432 32152 tablet_bootstrap.cc:492] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71: No bootstrap required, opened a new log
I20261115 21:02:19.482674 32152 ts_tablet_manager.cc:1449] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20261115 21:02:19.483137 32154 tablet_bootstrap.cc:492] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a: No bootstrap required, opened a new log
I20261115 21:02:19.483326 32154 ts_tablet_manager.cc:1449] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20261115 21:02:19.483711 32153 tablet_bootstrap.cc:654] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:19.484516 32153 log.cc:826] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:19.485639 32152 raft_consensus.cc:360] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } }
I20261115 21:02:19.485738 32154 raft_consensus.cc:360] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } }
I20261115 21:02:19.485845 32153 tablet_bootstrap.cc:492] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50: No bootstrap required, opened a new log
I20261115 21:02:19.485844 32152 raft_consensus.cc:748] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ae8c02d6e34647bca4953f59b3cf71, State: Initialized, Role: FOLLOWER
I20261115 21:02:19.485903 32154 raft_consensus.cc:748] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715f35676eb84845b460e696c9c7d27a, State: Initialized, Role: FOLLOWER
I20261115 21:02:19.486027 32153 ts_tablet_manager.cc:1449] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20261115 21:02:19.486164 32152 consensus_queue.cc:261] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } }
I20261115 21:02:19.486194 32154 consensus_queue.cc:261] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } }
I20261115 21:02:19.486793 32130 heartbeater.cc:500] Master 127.29.135.254:43903 was elected leader, sending a full tablet report...
I20261115 21:02:19.486915 32154 ts_tablet_manager.cc:1480] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20261115 21:02:19.486915 32152 ts_tablet_manager.cc:1480] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20261115 21:02:19.487260 31868 heartbeater.cc:500] Master 127.29.135.254:43903 was elected leader, sending a full tablet report...
I20261115 21:02:19.488703 32153 raft_consensus.cc:360] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } }
I20261115 21:02:19.488885 32153 raft_consensus.cc:748] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b367994359394faea71a66950795aa50, State: Initialized, Role: FOLLOWER
I20261115 21:02:19.489184 32153 consensus_queue.cc:261] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } }
I20261115 21:02:19.489768 31999 heartbeater.cc:500] Master 127.29.135.254:43903 was elected leader, sending a full tablet report...
I20261115 21:02:19.489847 32153 ts_tablet_manager.cc:1480] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20261115 21:02:19.510775 31822 tablet_service.cc:2033] Received Run Leader Election RPC: tablet_id: "cfe14815e02e4d6eab658ba9a0b98154"
dest_uuid: "715f35676eb84845b460e696c9c7d27a"
from {username='slave'} at 127.0.0.1:45464
I20261115 21:02:19.510944 31822 raft_consensus.cc:499] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20261115 21:02:19.511039 31822 raft_consensus.cc:3096] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:19.511843 31822 raft_consensus.cc:521] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } }
I20261115 21:02:19.512758 31822 leader_election.cc:290] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [CANDIDATE]: Term 1 election: Requested vote from peers a8ae8c02d6e34647bca4953f59b3cf71 (127.29.135.196:34143), b367994359394faea71a66950795aa50 (127.29.135.195:45161)
I20261115 21:02:19.516764 31953 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cfe14815e02e4d6eab658ba9a0b98154" candidate_uuid: "715f35676eb84845b460e696c9c7d27a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b367994359394faea71a66950795aa50"
I20261115 21:02:19.516772 32084 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cfe14815e02e4d6eab658ba9a0b98154" candidate_uuid: "715f35676eb84845b460e696c9c7d27a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a8ae8c02d6e34647bca4953f59b3cf71"
I20261115 21:02:19.516937 31953 raft_consensus.cc:3096] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:19.516940 32084 raft_consensus.cc:3096] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:19.517983 31953 raft_consensus.cc:2490] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 715f35676eb84845b460e696c9c7d27a in term 1.
I20261115 21:02:19.518007 32084 raft_consensus.cc:2490] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 715f35676eb84845b460e696c9c7d27a in term 1.
I20261115 21:02:19.518404 31757 leader_election.cc:304] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 715f35676eb84845b460e696c9c7d27a, b367994359394faea71a66950795aa50; no voters:
I20261115 21:02:19.518709 32159 raft_consensus.cc:2826] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:19.518919 32159 raft_consensus.cc:705] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 1 LEADER]: Becoming Leader. State: Replica: 715f35676eb84845b460e696c9c7d27a, State: Running, Role: LEADER
I20261115 21:02:19.519258 32159 consensus_queue.cc:238] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } }
I20261115 21:02:19.524629 32159 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:19.524994 32159 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:19.528431 31544 catalog_manager.cc:5840] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a reported cstate change: term changed from 0 to 1, leader changed from <none> to 715f35676eb84845b460e696c9c7d27a (127.29.135.194). New cstate: current_term: 1 leader_uuid: "715f35676eb84845b460e696c9c7d27a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } health_report { overall_health: HEALTHY } } }
I20261115 21:02:19.576385 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 31518
I20261115 21:02:19.584956 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:19.585001 30239 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }
I20261115 21:02:19.587569 31822 consensus_queue.cc:238] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }
I20261115 21:02:19.588781 32083 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 FOLLOWER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20261115 21:02:19.588971 31953 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 FOLLOWER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20261115 21:02:19.589128 32169 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20261115 21:02:19.589324 32159 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20261115 21:02:19.590801 32159 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index -1 to 8, VOTER 7403415722954f7d8621d306bb174044 (127.29.135.193) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:19.591184 31953 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index -1 to 8, VOTER 7403415722954f7d8621d306bb174044 (127.29.135.193) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
W20261115 21:02:19.593791 31999 heartbeater.cc:648] Failed to heartbeat to 127.29.135.254:43903 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.135.254:43903: connect: Connection refused (error 111)
W20261115 21:02:19.593997 31868 heartbeater.cc:648] Failed to heartbeat to 127.29.135.254:43903 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.135.254:43903: connect: Connection refused (error 111)
I20261115 21:02:19.594002 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:19.594043 30239 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
I20261115 21:02:19.594103 30239 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } }
I20261115 21:02:19.594429 32083 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index -1 to 8, VOTER 7403415722954f7d8621d306bb174044 (127.29.135.193) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:19.594859 31822 consensus_queue.cc:238] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }
W20261115 21:02:19.596192 31755 consensus_peers.cc:597] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a -> Peer 7403415722954f7d8621d306bb174044 (127.29.135.193:40609): Couldn't send request to peer 7403415722954f7d8621d306bb174044. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfe14815e02e4d6eab658ba9a0b98154. This is attempt 1: this message will repeat every 5th retry.
I20261115 21:02:19.596541 31953 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 FOLLOWER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
W20261115 21:02:19.596750 32130 heartbeater.cc:648] Failed to heartbeat to 127.29.135.254:43903 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.135.254:43903: connect: Connection refused (error 111)
I20261115 21:02:19.596757 32083 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 FOLLOWER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20261115 21:02:19.596923 32167 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20261115 21:02:19.597105 32169 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20261115 21:02:19.598219 32169 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b367994359394faea71a66950795aa50 (127.29.135.195) changed from VOTER to NON_VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:19.598627 32083 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b367994359394faea71a66950795aa50 (127.29.135.195) changed from VOTER to NON_VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:19.600077 31953 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 LEARNER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b367994359394faea71a66950795aa50 (127.29.135.195) changed from VOTER to NON_VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:19.600792 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:19.600831 30239 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } }
I20261115 21:02:19.601518 31822 consensus_queue.cc:238] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }
I20261115 21:02:19.602595 32083 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 FOLLOWER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20261115 21:02:19.602612 31953 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 LEARNER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
W20261115 21:02:19.602842 31755 consensus_peers.cc:597] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a -> Peer 7403415722954f7d8621d306bb174044 (127.29.135.193:40609): Couldn't send request to peer 7403415722954f7d8621d306bb174044. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfe14815e02e4d6eab658ba9a0b98154. This is attempt 1: this message will repeat every 5th retry.
I20261115 21:02:19.602916 32174 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20261115 21:02:19.603073 32169 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20261115 21:02:19.604266 32175 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:19.604640 32083 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:19.604640 31953 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 LEARNER]: Committing config change with OpId 1.10: config changed from index 9 to 10. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:19.606544 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:19.606591 30239 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
I20261115 21:02:19.606657 30239 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: NON_VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } }
I20261115 21:02:19.607568 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:19.607611 30239 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: NON_VOTER last_known_addr { host: "127.29.135.194" port: 38899 } attrs { promote: false replace: false } }
I20261115 21:02:19.608558 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:19.608594 30239 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: true replace: false } }
I20261115 21:02:19.609508 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:19.609544 30239 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
I20261115 21:02:19.609591 30239 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } }
I20261115 21:02:19.610186 31822 consensus_queue.cc:238] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }
W20261115 21:02:19.611073 31755 consensus_peers.cc:597] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a -> Peer 7403415722954f7d8621d306bb174044 (127.29.135.193:40609): Couldn't send request to peer 7403415722954f7d8621d306bb174044. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfe14815e02e4d6eab658ba9a0b98154. This is attempt 1: this message will repeat every 5th retry.
I20261115 21:02:19.864897 31737 heartbeater.cc:500] Master 127.29.135.254:43903 was elected leader, sending a full tablet report...
W20261115 21:02:19.865939 31737 heartbeater.cc:648] Failed to heartbeat to 127.29.135.254:43903 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.135.254:43903: connect: Connection refused (error 111)
I20261115 21:02:20.176393 32194 ts_tablet_manager.cc:979] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: Initiating tablet copy from peer 715f35676eb84845b460e696c9c7d27a (127.29.135.194:38899)
I20261115 21:02:20.176986 32194 tablet_copy_client.cc:323] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: tablet copy: Beginning tablet copy session from remote peer at address 127.29.135.194:38899
I20261115 21:02:20.180460 31842 tablet_copy_service.cc:140] P 715f35676eb84845b460e696c9c7d27a: Received BeginTabletCopySession request for tablet cfe14815e02e4d6eab658ba9a0b98154 from peer 7403415722954f7d8621d306bb174044 ({username='slave'} at 127.29.135.193:42965)
I20261115 21:02:20.180574 31842 tablet_copy_service.cc:161] P 715f35676eb84845b460e696c9c7d27a: Beginning new tablet copy session on tablet cfe14815e02e4d6eab658ba9a0b98154 from peer 7403415722954f7d8621d306bb174044 at {username='slave'} at 127.29.135.193:42965: session id = 7403415722954f7d8621d306bb174044-cfe14815e02e4d6eab658ba9a0b98154
I20261115 21:02:20.181850 31842 tablet_copy_source_session.cc:215] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a: Tablet Copy: opened 0 blocks and 1 log segments
I20261115 21:02:20.185287 32194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfe14815e02e4d6eab658ba9a0b98154. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:20.189838 32194 tablet_copy_client.cc:806] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: tablet copy: Starting download of 0 data blocks...
I20261115 21:02:20.190018 32194 tablet_copy_client.cc:670] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: tablet copy: Starting download of 1 WAL segments...
I20261115 21:02:20.191584 32194 tablet_copy_client.cc:538] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261115 21:02:20.193027 32194 tablet_bootstrap.cc:492] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: Bootstrap starting.
I20261115 21:02:20.198382 32194 log.cc:826] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:20.211918 32194 tablet_bootstrap.cc:492] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=10 ignored=0} inserts{seen=300 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20261115 21:02:20.212195 32194 tablet_bootstrap.cc:492] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: Bootstrap complete.
I20261115 21:02:20.212448 32194 ts_tablet_manager.cc:1449] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: Time spent bootstrapping tablet: real 0.020s user 0.019s sys 0.000s
I20261115 21:02:20.214464 32194 raft_consensus.cc:360] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }
I20261115 21:02:20.214915 32194 raft_consensus.cc:748] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7403415722954f7d8621d306bb174044, State: Initialized, Role: FOLLOWER
I20261115 21:02:20.215197 32194 consensus_queue.cc:261] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }
I20261115 21:02:20.215863 32194 ts_tablet_manager.cc:1480] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20261115 21:02:20.216507 31842 tablet_copy_service.cc:342] P 715f35676eb84845b460e696c9c7d27a: Request end of tablet copy session 7403415722954f7d8621d306bb174044-cfe14815e02e4d6eab658ba9a0b98154 received from {username='slave'} at 127.29.135.193:42965
I20261115 21:02:20.216609 31842 tablet_copy_service.cc:434] P 715f35676eb84845b460e696c9c7d27a: ending tablet copy session 7403415722954f7d8621d306bb174044-cfe14815e02e4d6eab658ba9a0b98154 on tablet cfe14815e02e4d6eab658ba9a0b98154 with peer 7403415722954f7d8621d306bb174044
I20261115 21:02:20.588136 31691 raft_consensus.cc:1227] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20261115 21:02:20.589229 32199 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, VOTER a8ae8c02d6e34647bca4953f59b3cf71 (127.29.135.196) evicted, NON_VOTER b367994359394faea71a66950795aa50 (127.29.135.195) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:20.589759 31691 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, VOTER a8ae8c02d6e34647bca4953f59b3cf71 (127.29.135.196) evicted, NON_VOTER b367994359394faea71a66950795aa50 (127.29.135.195) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:20.591960 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:20.592012 30239 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
I20261115 21:02:20.592078 30239 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } }
I20261115 21:02:20.593086 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:20.593128 30239 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
I20261115 21:02:20.593820 31822 consensus_queue.cc:238] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
I20261115 21:02:20.594944 31691 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044 [term 1 FOLLOWER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20261115 21:02:20.595009 31953 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 LEARNER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20261115 21:02:20.595382 32193 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20261115 21:02:20.595579 32199 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20261115 21:02:20.597059 31953 raft_consensus.cc:879] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }.
Pending config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }:
New config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
W20261115 21:02:20.597142 31953 raft_consensus.cc:1590] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 12' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }.
Pending config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }:
New config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
I20261115 21:02:20.597312 32200 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER b367994359394faea71a66950795aa50 (127.29.135.195) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } }
I20261115 21:02:20.597445 31953 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.11: config changed from index 10 to 11, VOTER a8ae8c02d6e34647bca4953f59b3cf71 (127.29.135.196) evicted, NON_VOTER b367994359394faea71a66950795aa50 (127.29.135.195) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:20.597775 31691 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER b367994359394faea71a66950795aa50 (127.29.135.195) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } }
I20261115 21:02:20.599637 31953 raft_consensus.cc:1491] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20261115 21:02:20.599969 31953 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER b367994359394faea71a66950795aa50 (127.29.135.195) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } }
I20261115 21:02:20.600219 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:20.600261 30239 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } }
I20261115 21:02:20.600905 31822 consensus_queue.cc:238] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } }
I20261115 21:02:20.602130 32083 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 FOLLOWER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20261115 21:02:20.602228 31691 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044 [term 1 FOLLOWER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20261115 21:02:20.602241 31953 raft_consensus.cc:1285] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 FOLLOWER]: Refusing update from remote peer 715f35676eb84845b460e696c9c7d27a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20261115 21:02:20.602572 32193 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20261115 21:02:20.602751 32204 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20261115 21:02:20.603024 32193 consensus_queue.cc:1049] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20261115 21:02:20.603926 32083 raft_consensus.cc:879] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }.
Pending config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }:
New config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
W20261115 21:02:20.603994 32083 raft_consensus.cc:1590] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 12' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: true replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: NON_VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } } peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }.
Pending config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }:
New config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
I20261115 21:02:20.604260 32083 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.11: config changed from index 10 to 11, VOTER a8ae8c02d6e34647bca4953f59b3cf71 (127.29.135.196) evicted, NON_VOTER b367994359394faea71a66950795aa50 (127.29.135.195) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } }
I20261115 21:02:20.604578 32193 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P 715f35676eb84845b460e696c9c7d27a [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a8ae8c02d6e34647bca4953f59b3cf71 (127.29.135.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } } }
I20261115 21:02:20.605906 31953 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P b367994359394faea71a66950795aa50 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a8ae8c02d6e34647bca4953f59b3cf71 (127.29.135.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } } }
I20261115 21:02:20.606561 31691 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P 7403415722954f7d8621d306bb174044 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a8ae8c02d6e34647bca4953f59b3cf71 (127.29.135.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } } }
I20261115 21:02:20.606808 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:20.606851 32083 raft_consensus.cc:1491] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20261115 21:02:20.606848 30239 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
I20261115 21:02:20.606915 30239 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } }
I20261115 21:02:20.607136 32083 raft_consensus.cc:879] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }.
Pending config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }:
New config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } }
W20261115 21:02:20.607185 32083 raft_consensus.cc:1590] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 13' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } }.
Pending config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }:
New config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } }
I20261115 21:02:20.607350 32083 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER b367994359394faea71a66950795aa50 (127.29.135.195) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } }
I20261115 21:02:20.609407 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:20.609454 30239 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } }
I20261115 21:02:20.610376 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:20.611020 32083 raft_consensus.cc:1491] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20261115 21:02:20.611232 30239 raft_config_change-itest.cc:349] submitting config change with changes:
I20261115 21:02:20.611279 30239 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: true } }
I20261115 21:02:20.611333 30239 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: true replace: false } }
I20261115 21:02:20.611325 32083 raft_consensus.cc:2983] T cfe14815e02e4d6eab658ba9a0b98154 P a8ae8c02d6e34647bca4953f59b3cf71 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a8ae8c02d6e34647bca4953f59b3cf71 (127.29.135.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "715f35676eb84845b460e696c9c7d27a" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 38899 } } peers { permanent_uuid: "7403415722954f7d8621d306bb174044" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40609 } attrs { promote: false replace: false } } peers { permanent_uuid: "b367994359394faea71a66950795aa50" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 45161 } attrs { promote: false replace: false } } peers { permanent_uuid: "a8ae8c02d6e34647bca4953f59b3cf71" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 34143 } attrs { promote: false replace: false } } }
I20261115 21:02:20.612452 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 31609
I20261115 21:02:20.618582 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 31740
I20261115 21:02:20.623924 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 31871
I20261115 21:02:20.628821 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 32002
2026-11-15T21:02:20Z chronyd exiting
[ OK ] RaftConfigChangeITest.TestBulkChangeConfig (2198 ms)
[----------] 3 tests from RaftConfigChangeITest (14625 ms total)
[----------] 3 tests from SlowTabletBootstrapTest
[ RUN ] SlowTabletBootstrapTest.TestSlowBootstrap
I20261115 21:02:20.650071 30239 ts_itest-base.cc:115] Starting cluster with:
I20261115 21:02:20.650112 30239 ts_itest-base.cc:116] --------------
I20261115 21:02:20.650126 30239 ts_itest-base.cc:117] 4 tablet servers
I20261115 21:02:20.650149 30239 ts_itest-base.cc:118] 3 replicas per TS
I20261115 21:02:20.650172 30239 ts_itest-base.cc:119] --------------
2026-11-15T21:02:20Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-15T21:02:20Z Disabled control of system clock
I20261115 21:02:20.657284 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:40985
--webserver_interface=127.29.135.254
--webserver_port=0
--builtin_ntp_servers=127.29.135.212:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.135.254:40985 with env {}
W20261115 21:02:20.753625 32215 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:20.753779 32215 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:20.753834 32215 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:20.757104 32215 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261115 21:02:20.757157 32215 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:20.757185 32215 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261115 21:02:20.757211 32215 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261115 21:02:20.761210 32215 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.135.254:40985
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:40985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.29.135.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.32215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:20.761915 32215 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:20.762605 32215 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:20.767287 32223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:20.767294 32220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:20.767474 32215 server_base.cc:1061] running on GCE node
W20261115 21:02:20.767354 32221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:20.767910 32215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:20.768553 32215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:20.769694 32215 hybrid_clock.cc:648] HybridClock initialized: now 1794776540769674 us; error 31 us; skew 500 ppm
I20261115 21:02:20.771193 32215 webserver.cc:577] Webserver started at http://127.29.135.254:39523/ using document root <none> and password file <none>
I20261115 21:02:20.771698 32215 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:20.771742 32215 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:20.771925 32215 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:20.773096 32215 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed"
format_stamp: "Formatted at 2026-11-15 21:02:20 on dist-test-slave-xh31"
I20261115 21:02:20.773468 32215 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed"
format_stamp: "Formatted at 2026-11-15 21:02:20 on dist-test-slave-xh31"
I20261115 21:02:20.776474 32215 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20261115 21:02:20.778576 32229 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:20.779454 32215 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:20.779574 32215 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed"
format_stamp: "Formatted at 2026-11-15 21:02:20 on dist-test-slave-xh31"
I20261115 21:02:20.779652 32215 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:20.793926 32215 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:20.794371 32215 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:20.794569 32215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:20.801081 32215 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.254:40985
I20261115 21:02:20.801095 32281 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.254:40985 every 8 connection(s)
I20261115 21:02:20.802044 32215 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20261115 21:02:20.805634 32282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:20.811491 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 32215
I20261115 21:02:20.811626 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal/instance
I20261115 21:02:20.811884 32282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed: Bootstrap starting.
I20261115 21:02:20.814404 32282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:20.815215 32282 log.cc:826] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:20.816617 32282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed: No bootstrap required, opened a new log
I20261115 21:02:20.819507 32282 raft_consensus.cc:360] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40985 } }
I20261115 21:02:20.819684 32282 raft_consensus.cc:386] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:20.819725 32282 raft_consensus.cc:748] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb6de54f4634baea3e1e6ee4cd1c3ed, State: Initialized, Role: FOLLOWER
I20261115 21:02:20.820060 32282 consensus_queue.cc:261] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40985 } }
I20261115 21:02:20.820168 32282 raft_consensus.cc:400] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261115 21:02:20.820209 32282 raft_consensus.cc:499] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261115 21:02:20.820267 32282 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:20.821146 32282 raft_consensus.cc:521] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40985 } }
I20261115 21:02:20.821480 32282 leader_election.cc:304] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cb6de54f4634baea3e1e6ee4cd1c3ed; no voters:
I20261115 21:02:20.821715 32282 leader_election.cc:290] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20261115 21:02:20.821830 32287 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:20.822024 32287 raft_consensus.cc:705] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [term 1 LEADER]: Becoming Leader. State: Replica: 4cb6de54f4634baea3e1e6ee4cd1c3ed, State: Running, Role: LEADER
I20261115 21:02:20.822306 32287 consensus_queue.cc:238] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40985 } }
I20261115 21:02:20.822732 32282 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [sys.catalog]: configured and running, proceeding with master startup.
I20261115 21:02:20.823786 32289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cb6de54f4634baea3e1e6ee4cd1c3ed. Latest consensus state: current_term: 1 leader_uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40985 } } }
I20261115 21:02:20.823897 32289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:20.824242 32296 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261115 21:02:20.824750 32288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb6de54f4634baea3e1e6ee4cd1c3ed" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 40985 } } }
I20261115 21:02:20.824842 32288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:20.827450 32296 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261115 21:02:20.832280 32296 catalog_manager.cc:1398] Generated new cluster ID: 97ea4d5729bd42dd8f85bb53447545a2
I20261115 21:02:20.832342 32296 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261115 21:02:20.842931 32296 catalog_manager.cc:1421] Generated new certificate authority record
I20261115 21:02:20.843627 32296 catalog_manager.cc:1555] Loading token signing keys...
I20261115 21:02:20.848212 32296 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 4cb6de54f4634baea3e1e6ee4cd1c3ed: Generated new TSK 0
I20261115 21:02:20.848676 32296 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261115 21:02:20.852638 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.193:0
--local_ip_for_outbound_sockets=127.29.135.193
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40985
--builtin_ntp_servers=127.29.135.212:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261115 21:02:20.947566 32306 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:20.947736 32306 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:20.947758 32306 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:20.947822 32306 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:20.951171 32306 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:20.951216 32306 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:20.951249 32306 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:20.951324 32306 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:20.951364 32306 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.193
I20261115 21:02:20.955148 32306 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40985
--never_fsync=true
--heap_profile_path=/tmp/kudu.32306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:20.955814 32306 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:20.956465 32306 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:20.961567 32311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:20.961607 32314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:20.961726 32306 server_base.cc:1061] running on GCE node
W20261115 21:02:20.961886 32312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:20.962287 32306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:20.962931 32306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:20.964100 32306 hybrid_clock.cc:648] HybridClock initialized: now 1794776540964080 us; error 32 us; skew 500 ppm
I20261115 21:02:20.965868 32306 webserver.cc:577] Webserver started at http://127.29.135.193:43841/ using document root <none> and password file <none>
I20261115 21:02:20.966542 32306 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:20.966605 32306 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:20.966846 32306 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:20.968374 32306 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "66e5d766917848a3bb9927e7126c5112"
format_stamp: "Formatted at 2026-11-15 21:02:20 on dist-test-slave-xh31"
I20261115 21:02:20.968855 32306 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "66e5d766917848a3bb9927e7126c5112"
format_stamp: "Formatted at 2026-11-15 21:02:20 on dist-test-slave-xh31"
I20261115 21:02:20.972785 32306 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20261115 21:02:20.975312 32320 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:20.976202 32306 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20261115 21:02:20.976294 32306 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "66e5d766917848a3bb9927e7126c5112"
format_stamp: "Formatted at 2026-11-15 21:02:20 on dist-test-slave-xh31"
I20261115 21:02:20.976374 32306 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:20.986934 32306 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:20.987275 32306 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:20.987394 32306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:20.987808 32306 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:20.988651 32306 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:20.988694 32306 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:20.988747 32306 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:20.988775 32306 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:20.996573 32306 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.193:37285
I20261115 21:02:20.996609 32433 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.193:37285 every 8 connection(s)
I20261115 21:02:20.997430 32306 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20261115 21:02:21.004419 32434 heartbeater.cc:345] Connected to a master server at 127.29.135.254:40985
I20261115 21:02:21.004602 32434 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:21.005014 32434 heartbeater.cc:508] Master 127.29.135.254:40985 requested a full tablet report, sending...
I20261115 21:02:21.006417 32246 ts_manager.cc:210] Registered new tserver with Master: 66e5d766917848a3bb9927e7126c5112 (127.29.135.193:37285)
I20261115 21:02:21.007344 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 32306
I20261115 21:02:21.007470 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20261115 21:02:21.007906 32246 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.193:48735
I20261115 21:02:21.009584 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.194:0
--local_ip_for_outbound_sockets=127.29.135.194
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40985
--builtin_ntp_servers=127.29.135.212:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261115 21:02:21.108850 32437 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:21.109067 32437 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:21.109115 32437 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:21.109189 32437 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:21.112511 32437 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:21.112566 32437 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:21.112627 32437 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:21.112737 32437 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:21.112782 32437 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.194
I20261115 21:02:21.116643 32437 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40985
--never_fsync=true
--heap_profile_path=/tmp/kudu.32437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:21.117395 32437 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:21.118103 32437 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:21.123607 32445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:21.123641 32442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:21.123648 32443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:21.123863 32437 server_base.cc:1061] running on GCE node
I20261115 21:02:21.124218 32437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:21.124719 32437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:21.125857 32437 hybrid_clock.cc:648] HybridClock initialized: now 1794776541125833 us; error 43 us; skew 500 ppm
I20261115 21:02:21.127540 32437 webserver.cc:577] Webserver started at http://127.29.135.194:33557/ using document root <none> and password file <none>
I20261115 21:02:21.128085 32437 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:21.128134 32437 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:21.128329 32437 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:21.129563 32437 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "a825236360a8483cb73b4d365f099dad"
format_stamp: "Formatted at 2026-11-15 21:02:21 on dist-test-slave-xh31"
I20261115 21:02:21.129972 32437 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "a825236360a8483cb73b4d365f099dad"
format_stamp: "Formatted at 2026-11-15 21:02:21 on dist-test-slave-xh31"
I20261115 21:02:21.132949 32437 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20261115 21:02:21.135085 32451 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:21.135931 32437 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20261115 21:02:21.136034 32437 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "a825236360a8483cb73b4d365f099dad"
format_stamp: "Formatted at 2026-11-15 21:02:21 on dist-test-slave-xh31"
I20261115 21:02:21.136116 32437 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:21.148093 32437 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:21.148454 32437 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:21.148578 32437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:21.149039 32437 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:21.150044 32437 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:21.150089 32437 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:21.150133 32437 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:21.150162 32437 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:21.158164 32437 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.194:37475
I20261115 21:02:21.158209 32564 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.194:37475 every 8 connection(s)
I20261115 21:02:21.159070 32437 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261115 21:02:21.164219 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 32437
I20261115 21:02:21.164332 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20261115 21:02:21.166046 32565 heartbeater.cc:345] Connected to a master server at 127.29.135.254:40985
I20261115 21:02:21.166236 32565 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:21.166450 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.195:0
--local_ip_for_outbound_sockets=127.29.135.195
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40985
--builtin_ntp_servers=127.29.135.212:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261115 21:02:21.166708 32565 heartbeater.cc:508] Master 127.29.135.254:40985 requested a full tablet report, sending...
I20261115 21:02:21.167635 32245 ts_manager.cc:210] Registered new tserver with Master: a825236360a8483cb73b4d365f099dad (127.29.135.194:37475)
I20261115 21:02:21.168356 32245 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.194:54285
W20261115 21:02:21.265487 32568 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:21.265659 32568 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:21.265679 32568 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:21.265722 32568 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:21.268980 32568 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:21.269024 32568 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:21.269057 32568 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:21.269121 32568 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:21.269148 32568 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.195
I20261115 21:02:21.272986 32568 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40985
--never_fsync=true
--heap_profile_path=/tmp/kudu.32568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:21.273638 32568 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:21.274250 32568 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:21.279310 32573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:21.279419 32576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:21.279553 32568 server_base.cc:1061] running on GCE node
W20261115 21:02:21.279529 32574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:21.279901 32568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:21.280438 32568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:21.281576 32568 hybrid_clock.cc:648] HybridClock initialized: now 1794776541281559 us; error 37 us; skew 500 ppm
I20261115 21:02:21.283071 32568 webserver.cc:577] Webserver started at http://127.29.135.195:46439/ using document root <none> and password file <none>
I20261115 21:02:21.283569 32568 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:21.283618 32568 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:21.283809 32568 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:21.285013 32568 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "42d9d1b29996412d9ff55347e3c665b3"
format_stamp: "Formatted at 2026-11-15 21:02:21 on dist-test-slave-xh31"
I20261115 21:02:21.285400 32568 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "42d9d1b29996412d9ff55347e3c665b3"
format_stamp: "Formatted at 2026-11-15 21:02:21 on dist-test-slave-xh31"
I20261115 21:02:21.288303 32568 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20261115 21:02:21.290331 32582 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:21.291270 32568 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20261115 21:02:21.291369 32568 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "42d9d1b29996412d9ff55347e3c665b3"
format_stamp: "Formatted at 2026-11-15 21:02:21 on dist-test-slave-xh31"
I20261115 21:02:21.291446 32568 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:21.303287 32568 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:21.303623 32568 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:21.303745 32568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:21.304168 32568 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:21.304993 32568 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:21.305037 32568 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:21.305086 32568 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:21.305114 32568 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:21.312849 32568 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.195:34295
I20261115 21:02:21.312898 32695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.195:34295 every 8 connection(s)
I20261115 21:02:21.313680 32568 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261115 21:02:21.320571 32696 heartbeater.cc:345] Connected to a master server at 127.29.135.254:40985
I20261115 21:02:21.320767 32696 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:21.321218 32696 heartbeater.cc:508] Master 127.29.135.254:40985 requested a full tablet report, sending...
I20261115 21:02:21.321388 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 32568
I20261115 21:02:21.321501 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20261115 21:02:21.322327 32245 ts_manager.cc:210] Registered new tserver with Master: 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295)
I20261115 21:02:21.323009 32245 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.195:46587
I20261115 21:02:21.323635 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.196:0
--local_ip_for_outbound_sockets=127.29.135.196
--webserver_interface=127.29.135.196
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40985
--builtin_ntp_servers=127.29.135.212:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261115 21:02:21.420423 32699 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:21.420620 32699 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:21.420642 32699 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:21.420689 32699 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:21.424149 32699 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:21.424198 32699 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:21.424233 32699 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:21.424368 32699 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:21.424405 32699 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.196
I20261115 21:02:21.428335 32699 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.29.135.196
--webserver_port=0
--tserver_master_addrs=127.29.135.254:40985
--never_fsync=true
--heap_profile_path=/tmp/kudu.32699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.196
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:21.429035 32699 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:21.429688 32699 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:21.434870 32705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:21.434870 32704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:21.434899 32707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:21.435070 32699 server_base.cc:1061] running on GCE node
I20261115 21:02:21.435459 32699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:21.435940 32699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:21.437086 32699 hybrid_clock.cc:648] HybridClock initialized: now 1794776541437069 us; error 32 us; skew 500 ppm
I20261115 21:02:21.438654 32699 webserver.cc:577] Webserver started at http://127.29.135.196:38309/ using document root <none> and password file <none>
I20261115 21:02:21.439157 32699 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:21.439198 32699 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:21.439383 32699 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:21.440621 32699 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "d95bd74c3ca642588f861055ca948151"
format_stamp: "Formatted at 2026-11-15 21:02:21 on dist-test-slave-xh31"
I20261115 21:02:21.440991 32699 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "d95bd74c3ca642588f861055ca948151"
format_stamp: "Formatted at 2026-11-15 21:02:21 on dist-test-slave-xh31"
I20261115 21:02:21.443964 32699 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20261115 21:02:21.445976 32713 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:21.446856 32699 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:21.446949 32699 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "d95bd74c3ca642588f861055ca948151"
format_stamp: "Formatted at 2026-11-15 21:02:21 on dist-test-slave-xh31"
I20261115 21:02:21.447019 32699 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:21.465651 32699 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:21.466135 32699 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:21.466262 32699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:21.466775 32699 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:21.467762 32699 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:21.467811 32699 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:21.467854 32699 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:21.467873 32699 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:21.476493 32699 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.196:38489
I20261115 21:02:21.476534 358 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.196:38489 every 8 connection(s)
I20261115 21:02:21.477499 32699 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261115 21:02:21.478740 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 32699
I20261115 21:02:21.478871 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20261115 21:02:21.485774 359 heartbeater.cc:345] Connected to a master server at 127.29.135.254:40985
I20261115 21:02:21.486029 359 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:21.486616 359 heartbeater.cc:508] Master 127.29.135.254:40985 requested a full tablet report, sending...
I20261115 21:02:21.487658 32245 ts_manager.cc:210] Registered new tserver with Master: d95bd74c3ca642588f861055ca948151 (127.29.135.196:38489)
I20261115 21:02:21.488236 32245 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.196:50543
I20261115 21:02:21.493257 30239 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261115 21:02:21.501756 32245 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53282:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20261115 21:02:21.519894 32499 tablet_service.cc:1500] Processing CreateTablet for tablet ebafa55443604f13824f211c1dcc6e41 (DEFAULT_TABLE table=TestTable [id=07c1fb4a5c734d9abfe1ca42792b775f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:21.519898 32630 tablet_service.cc:1500] Processing CreateTablet for tablet ebafa55443604f13824f211c1dcc6e41 (DEFAULT_TABLE table=TestTable [id=07c1fb4a5c734d9abfe1ca42792b775f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:21.519899 32761 tablet_service.cc:1500] Processing CreateTablet for tablet ebafa55443604f13824f211c1dcc6e41 (DEFAULT_TABLE table=TestTable [id=07c1fb4a5c734d9abfe1ca42792b775f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:21.520749 32499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebafa55443604f13824f211c1dcc6e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:21.520769 32630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebafa55443604f13824f211c1dcc6e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:21.520769 32761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebafa55443604f13824f211c1dcc6e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:21.525915 376 tablet_bootstrap.cc:492] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad: Bootstrap starting.
I20261115 21:02:21.526961 378 tablet_bootstrap.cc:492] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Bootstrap starting.
I20261115 21:02:21.528594 376 tablet_bootstrap.cc:654] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:21.529237 378 tablet_bootstrap.cc:654] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:21.529531 376 log.cc:826] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:21.530082 378 log.cc:826] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:21.530231 377 tablet_bootstrap.cc:492] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151: Bootstrap starting.
I20261115 21:02:21.531036 376 tablet_bootstrap.cc:492] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad: No bootstrap required, opened a new log
I20261115 21:02:21.531231 376 ts_tablet_manager.cc:1449] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20261115 21:02:21.531548 378 tablet_bootstrap.cc:492] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: No bootstrap required, opened a new log
I20261115 21:02:21.531741 378 ts_tablet_manager.cc:1449] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20261115 21:02:21.532500 377 tablet_bootstrap.cc:654] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:21.533301 377 log.cc:826] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:21.533802 376 raft_consensus.cc:360] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:21.534056 376 raft_consensus.cc:386] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:21.534118 376 raft_consensus.cc:748] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a825236360a8483cb73b4d365f099dad, State: Initialized, Role: FOLLOWER
I20261115 21:02:21.534524 376 consensus_queue.cc:261] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:21.534824 377 tablet_bootstrap.cc:492] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151: No bootstrap required, opened a new log
I20261115 21:02:21.534786 378 raft_consensus.cc:360] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:21.534976 378 raft_consensus.cc:386] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:21.535012 377 ts_tablet_manager.cc:1449] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20261115 21:02:21.535175 32565 heartbeater.cc:500] Master 127.29.135.254:40985 was elected leader, sending a full tablet report...
I20261115 21:02:21.535275 376 ts_tablet_manager.cc:1480] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20261115 21:02:21.535023 378 raft_consensus.cc:748] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42d9d1b29996412d9ff55347e3c665b3, State: Initialized, Role: FOLLOWER
I20261115 21:02:21.536610 378 consensus_queue.cc:261] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:21.537240 32696 heartbeater.cc:500] Master 127.29.135.254:40985 was elected leader, sending a full tablet report...
I20261115 21:02:21.537356 378 ts_tablet_manager.cc:1480] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20261115 21:02:21.538909 377 raft_consensus.cc:360] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:21.539134 377 raft_consensus.cc:386] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:21.539201 377 raft_consensus.cc:748] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95bd74c3ca642588f861055ca948151, State: Initialized, Role: FOLLOWER
I20261115 21:02:21.539594 377 consensus_queue.cc:261] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:21.540273 377 ts_tablet_manager.cc:1480] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20261115 21:02:21.540357 359 heartbeater.cc:500] Master 127.29.135.254:40985 was elected leader, sending a full tablet report...
W20261115 21:02:21.560384 32566 tablet.cc:2441] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261115 21:02:21.578281 360 tablet.cc:2441] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261115 21:02:21.584669 383 raft_consensus.cc:499] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261115 21:02:21.584830 383 raft_consensus.cc:521] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:21.585907 383 leader_election.cc:290] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a825236360a8483cb73b4d365f099dad (127.29.135.194:37475), d95bd74c3ca642588f861055ca948151 (127.29.135.196:38489)
I20261115 21:02:21.590955 312 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "42d9d1b29996412d9ff55347e3c665b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d95bd74c3ca642588f861055ca948151" is_pre_election: true
I20261115 21:02:21.591007 32519 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "42d9d1b29996412d9ff55347e3c665b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a825236360a8483cb73b4d365f099dad" is_pre_election: true
I20261115 21:02:21.591279 312 raft_consensus.cc:2490] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42d9d1b29996412d9ff55347e3c665b3 in term 0.
I20261115 21:02:21.591279 32519 raft_consensus.cc:2490] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42d9d1b29996412d9ff55347e3c665b3 in term 0.
I20261115 21:02:21.591753 32585 leader_election.cc:304] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42d9d1b29996412d9ff55347e3c665b3, d95bd74c3ca642588f861055ca948151; no voters:
I20261115 21:02:21.592041 383 raft_consensus.cc:2826] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261115 21:02:21.592110 383 raft_consensus.cc:499] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261115 21:02:21.592164 383 raft_consensus.cc:3096] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:21.593048 383 raft_consensus.cc:521] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:21.593379 383 leader_election.cc:290] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [CANDIDATE]: Term 1 election: Requested vote from peers a825236360a8483cb73b4d365f099dad (127.29.135.194:37475), d95bd74c3ca642588f861055ca948151 (127.29.135.196:38489)
I20261115 21:02:21.593724 32519 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "42d9d1b29996412d9ff55347e3c665b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a825236360a8483cb73b4d365f099dad"
I20261115 21:02:21.593720 312 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "42d9d1b29996412d9ff55347e3c665b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d95bd74c3ca642588f861055ca948151"
I20261115 21:02:21.593850 312 raft_consensus.cc:3096] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:21.593849 32519 raft_consensus.cc:3096] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:21.594913 312 raft_consensus.cc:2490] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42d9d1b29996412d9ff55347e3c665b3 in term 1.
I20261115 21:02:21.594931 32519 raft_consensus.cc:2490] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42d9d1b29996412d9ff55347e3c665b3 in term 1.
I20261115 21:02:21.595197 32585 leader_election.cc:304] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42d9d1b29996412d9ff55347e3c665b3, d95bd74c3ca642588f861055ca948151; no voters:
I20261115 21:02:21.595340 383 raft_consensus.cc:2826] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:21.595548 383 raft_consensus.cc:705] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 1 LEADER]: Becoming Leader. State: Replica: 42d9d1b29996412d9ff55347e3c665b3, State: Running, Role: LEADER
I20261115 21:02:21.595893 383 consensus_queue.cc:238] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:21.599120 383 consensus_queue.cc:1049] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:21.599256 386 consensus_queue.cc:1049] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:21.599164 32245 catalog_manager.cc:5840] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195). New cstate: current_term: 1 leader_uuid: "42d9d1b29996412d9ff55347e3c665b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } health_report { overall_health: UNKNOWN } } }
I20261115 21:02:21.609712 30239 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261115 21:02:21.611092 30239 ts_itest-base.cc:246] Waiting for 1 tablets on tserver a825236360a8483cb73b4d365f099dad to finish bootstrapping
I20261115 21:02:21.615993 30239 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 42d9d1b29996412d9ff55347e3c665b3 to finish bootstrapping
I20261115 21:02:21.619988 30239 ts_itest-base.cc:246] Waiting for 1 tablets on tserver d95bd74c3ca642588f861055ca948151 to finish bootstrapping
I20261115 21:02:21.624118 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 32568
I20261115 21:02:21.629892 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.195:34295
--local_ip_for_outbound_sockets=127.29.135.195
--tserver_master_addrs=127.29.135.254:40985
--webserver_port=46439
--webserver_interface=127.29.135.195
--builtin_ntp_servers=127.29.135.212:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000
--tablet_bootstrap_inject_latency_ms=4000 with env {}
W20261115 21:02:21.727627 395 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:21.727818 395 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:21.727855 395 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:21.727913 395 flags.cc:432] Enabled unsafe flag: --tablet_bootstrap_inject_latency_ms=4000
W20261115 21:02:21.727941 395 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:21.731257 395 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:21.731312 395 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:21.731359 395 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:21.731465 395 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:21.731503 395 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.195
I20261115 21:02:21.735450 395 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.195:34295
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.29.135.195
--webserver_port=46439
--tserver_master_addrs=127.29.135.254:40985
--tablet_bootstrap_inject_latency_ms=4000
--never_fsync=true
--heap_profile_path=/tmp/kudu.395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:21.736164 395 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:21.736816 395 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:21.742009 403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:21.742031 400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:21.742064 401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:21.742132 395 server_base.cc:1061] running on GCE node
I20261115 21:02:21.742561 395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:21.743048 395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:21.744241 395 hybrid_clock.cc:648] HybridClock initialized: now 1794776541744225 us; error 32 us; skew 500 ppm
I20261115 21:02:21.745754 395 webserver.cc:577] Webserver started at http://127.29.135.195:46439/ using document root <none> and password file <none>
I20261115 21:02:21.746281 395 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:21.746343 395 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:21.749313 395 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:21.751199 409 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:21.752117 395 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:21.752213 395 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "42d9d1b29996412d9ff55347e3c665b3"
format_stamp: "Formatted at 2026-11-15 21:02:21 on dist-test-slave-xh31"
I20261115 21:02:21.752545 395 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:21.765655 395 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:21.766026 395 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:21.766149 395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:21.766608 395 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:21.767616 416 ts_tablet_manager.cc:588] Loading tablet metadata (0/1 complete)
I20261115 21:02:21.770820 395 ts_tablet_manager.cc:631] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20261115 21:02:21.770874 395 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20261115 21:02:21.770944 395 ts_tablet_manager.cc:646] Registering tablets (0/1 complete)
I20261115 21:02:21.772344 395 ts_tablet_manager.cc:662] Registered 1 tablets
I20261115 21:02:21.772385 395 ts_tablet_manager.cc:641] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
W20261115 21:02:21.772403 416 ts_tablet_manager.cc:1428] Injecting 4000ms delay in tablet bootstrapping
I20261115 21:02:21.780287 395 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.195:34295
I20261115 21:02:21.780315 523 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.195:34295 every 8 connection(s)
I20261115 21:02:21.781200 395 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261115 21:02:21.784677 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 395
I20261115 21:02:21.788060 524 heartbeater.cc:345] Connected to a master server at 127.29.135.254:40985
I20261115 21:02:21.788232 524 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:21.788615 524 heartbeater.cc:508] Master 127.29.135.254:40985 requested a full tablet report, sending...
I20261115 21:02:21.789912 32246 ts_manager.cc:210] Re-registered known tserver with Master: 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295)
I20261115 21:02:21.790709 32246 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.195:53559
I20261115 21:02:22.009855 32434 heartbeater.cc:500] Master 127.29.135.254:40985 was elected leader, sending a full tablet report...
I20261115 21:02:22.792402 524 heartbeater.cc:500] Master 127.29.135.254:40985 was elected leader, sending a full tablet report...
I20261115 21:02:23.104959 527 raft_consensus.cc:499] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 42d9d1b29996412d9ff55347e3c665b3)
I20261115 21:02:23.105104 527 raft_consensus.cc:521] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:23.105144 528 raft_consensus.cc:499] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 42d9d1b29996412d9ff55347e3c665b3)
I20261115 21:02:23.105285 528 raft_consensus.cc:521] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:23.106184 528 leader_election.cc:290] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295), a825236360a8483cb73b4d365f099dad (127.29.135.194:37475)
I20261115 21:02:23.106184 527 leader_election.cc:290] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295), d95bd74c3ca642588f861055ca948151 (127.29.135.196:38489)
I20261115 21:02:23.110947 32519 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "d95bd74c3ca642588f861055ca948151" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a825236360a8483cb73b4d365f099dad" is_pre_election: true
I20261115 21:02:23.111014 312 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "a825236360a8483cb73b4d365f099dad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d95bd74c3ca642588f861055ca948151" is_pre_election: true
I20261115 21:02:23.111083 32519 raft_consensus.cc:2490] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d95bd74c3ca642588f861055ca948151 in term 1.
I20261115 21:02:23.111150 312 raft_consensus.cc:2490] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a825236360a8483cb73b4d365f099dad in term 1.
I20261115 21:02:23.111552 32717 leader_election.cc:304] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a825236360a8483cb73b4d365f099dad, d95bd74c3ca642588f861055ca948151; no voters:
I20261115 21:02:23.111552 32454 leader_election.cc:304] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a825236360a8483cb73b4d365f099dad, d95bd74c3ca642588f861055ca948151; no voters:
I20261115 21:02:23.111775 527 raft_consensus.cc:2826] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261115 21:02:23.111829 527 raft_consensus.cc:499] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 1 FOLLOWER]: Starting leader election (detected failure of leader 42d9d1b29996412d9ff55347e3c665b3)
I20261115 21:02:23.111866 527 raft_consensus.cc:3096] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 1 FOLLOWER]: Advancing to term 2
I20261115 21:02:23.111977 528 raft_consensus.cc:2826] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261115 21:02:23.112028 528 raft_consensus.cc:499] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 42d9d1b29996412d9ff55347e3c665b3)
I20261115 21:02:23.112064 528 raft_consensus.cc:3096] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 1 FOLLOWER]: Advancing to term 2
I20261115 21:02:23.112687 527 raft_consensus.cc:521] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:23.111950 478 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "d95bd74c3ca642588f861055ca948151" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "42d9d1b29996412d9ff55347e3c665b3" is_pre_election: true
I20261115 21:02:23.113013 527 leader_election.cc:290] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [CANDIDATE]: Term 2 election: Requested vote from peers 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295), d95bd74c3ca642588f861055ca948151 (127.29.135.196:38489)
I20261115 21:02:23.112998 528 raft_consensus.cc:521] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:23.111948 477 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "a825236360a8483cb73b4d365f099dad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "42d9d1b29996412d9ff55347e3c665b3" is_pre_election: true
I20261115 21:02:23.113313 528 leader_election.cc:290] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 2 election: Requested vote from peers 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295), a825236360a8483cb73b4d365f099dad (127.29.135.194:37475)
I20261115 21:02:23.113596 476 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "a825236360a8483cb73b4d365f099dad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "42d9d1b29996412d9ff55347e3c665b3"
I20261115 21:02:23.113590 312 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "a825236360a8483cb73b4d365f099dad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d95bd74c3ca642588f861055ca948151"
I20261115 21:02:23.113734 312 raft_consensus.cc:2415] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a825236360a8483cb73b4d365f099dad in current term 2: Already voted for candidate d95bd74c3ca642588f861055ca948151 in this term.
I20261115 21:02:23.113727 32519 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "d95bd74c3ca642588f861055ca948151" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a825236360a8483cb73b4d365f099dad"
I20261115 21:02:23.113875 32519 raft_consensus.cc:2415] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d95bd74c3ca642588f861055ca948151 in current term 2: Already voted for candidate a825236360a8483cb73b4d365f099dad in this term.
I20261115 21:02:23.113902 476 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "d95bd74c3ca642588f861055ca948151" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "42d9d1b29996412d9ff55347e3c665b3"
W20261115 21:02:23.114261 32453 leader_election.cc:343] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295): Illegal state: must be running to vote when last-logged opid is not known
W20261115 21:02:23.114434 32453 leader_election.cc:343] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295): Illegal state: must be running to vote when last-logged opid is not known
W20261115 21:02:23.113945 32715 leader_election.cc:343] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295): Illegal state: must be running to vote when last-logged opid is not known
W20261115 21:02:23.119549 32715 leader_election.cc:343] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295): Illegal state: must be running to vote when last-logged opid is not known
I20261115 21:02:23.119591 32454 leader_election.cc:304] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a825236360a8483cb73b4d365f099dad; no voters: 42d9d1b29996412d9ff55347e3c665b3, d95bd74c3ca642588f861055ca948151
I20261115 21:02:23.119616 32715 leader_election.cc:304] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d95bd74c3ca642588f861055ca948151; no voters: 42d9d1b29996412d9ff55347e3c665b3, a825236360a8483cb73b4d365f099dad
I20261115 21:02:23.119730 527 raft_consensus.cc:2771] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20261115 21:02:23.119807 528 raft_consensus.cc:2771] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20261115 21:02:24.628387 537 raft_consensus.cc:499] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261115 21:02:24.628497 537 raft_consensus.cc:521] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:24.628840 537 leader_election.cc:290] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295), a825236360a8483cb73b4d365f099dad (127.29.135.194:37475)
I20261115 21:02:24.629200 32519 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "d95bd74c3ca642588f861055ca948151" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a825236360a8483cb73b4d365f099dad" is_pre_election: true
I20261115 21:02:24.629215 476 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "d95bd74c3ca642588f861055ca948151" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "42d9d1b29996412d9ff55347e3c665b3" is_pre_election: true
I20261115 21:02:24.629319 32519 raft_consensus.cc:2490] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d95bd74c3ca642588f861055ca948151 in term 2.
I20261115 21:02:24.629614 32717 leader_election.cc:304] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a825236360a8483cb73b4d365f099dad, d95bd74c3ca642588f861055ca948151; no voters:
W20261115 21:02:24.629724 32715 leader_election.cc:343] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295): Illegal state: must be running to vote when last-logged opid is not known
I20261115 21:02:24.629774 537 raft_consensus.cc:2826] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20261115 21:02:24.629817 537 raft_consensus.cc:499] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261115 21:02:24.629858 537 raft_consensus.cc:3096] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 2 FOLLOWER]: Advancing to term 3
I20261115 21:02:24.630661 537 raft_consensus.cc:521] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:24.630949 537 leader_election.cc:290] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 3 election: Requested vote from peers 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295), a825236360a8483cb73b4d365f099dad (127.29.135.194:37475)
I20261115 21:02:24.631253 476 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "d95bd74c3ca642588f861055ca948151" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "42d9d1b29996412d9ff55347e3c665b3"
I20261115 21:02:24.631273 32519 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ebafa55443604f13824f211c1dcc6e41" candidate_uuid: "d95bd74c3ca642588f861055ca948151" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a825236360a8483cb73b4d365f099dad"
I20261115 21:02:24.631361 32519 raft_consensus.cc:3096] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 2 FOLLOWER]: Advancing to term 3
W20261115 21:02:24.631564 32715 leader_election.cc:343] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295): Illegal state: must be running to vote when last-logged opid is not known
I20261115 21:02:24.632130 32519 raft_consensus.cc:2490] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d95bd74c3ca642588f861055ca948151 in term 3.
I20261115 21:02:24.632345 32717 leader_election.cc:304] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a825236360a8483cb73b4d365f099dad, d95bd74c3ca642588f861055ca948151; no voters: 42d9d1b29996412d9ff55347e3c665b3
I20261115 21:02:24.632490 537 raft_consensus.cc:2826] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 3 FOLLOWER]: Leader election won for term 3
I20261115 21:02:24.632648 537 raft_consensus.cc:705] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [term 3 LEADER]: Becoming Leader. State: Replica: d95bd74c3ca642588f861055ca948151, State: Running, Role: LEADER
I20261115 21:02:24.632988 537 consensus_queue.cc:238] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:24.634846 32519 raft_consensus.cc:1285] T ebafa55443604f13824f211c1dcc6e41 P a825236360a8483cb73b4d365f099dad [term 3 FOLLOWER]: Refusing update from remote peer d95bd74c3ca642588f861055ca948151: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20261115 21:02:24.635155 32246 catalog_manager.cc:5840] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 reported cstate change: term changed from 1 to 3, leader changed from 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195) to d95bd74c3ca642588f861055ca948151 (127.29.135.196). New cstate: current_term: 3 leader_uuid: "d95bd74c3ca642588f861055ca948151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } health_report { overall_health: HEALTHY } } }
I20261115 21:02:24.635257 537 consensus_queue.cc:1049] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20261115 21:02:24.635537 32715 consensus_peers.cc:597] T ebafa55443604f13824f211c1dcc6e41 P d95bd74c3ca642588f861055ca948151 -> Peer 42d9d1b29996412d9ff55347e3c665b3 (127.29.135.195:34295): Couldn't send request to peer 42d9d1b29996412d9ff55347e3c665b3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20261115 21:02:25.772881 416 tablet_bootstrap.cc:492] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Bootstrap starting.
I20261115 21:02:25.779569 416 log.cc:826] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:25.781898 416 tablet_bootstrap.cc:492] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261115 21:02:25.782217 416 tablet_bootstrap.cc:492] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Bootstrap complete.
I20261115 21:02:25.782653 416 ts_tablet_manager.cc:1449] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20261115 21:02:25.784852 416 raft_consensus.cc:360] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:25.785173 416 raft_consensus.cc:748] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42d9d1b29996412d9ff55347e3c665b3, State: Initialized, Role: FOLLOWER
I20261115 21:02:25.785523 416 consensus_queue.cc:261] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42d9d1b29996412d9ff55347e3c665b3" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 34295 } } peers { permanent_uuid: "a825236360a8483cb73b4d365f099dad" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 37475 } } peers { permanent_uuid: "d95bd74c3ca642588f861055ca948151" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 38489 } }
I20261115 21:02:25.786149 416 ts_tablet_manager.cc:1480] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20261115 21:02:26.090209 476 raft_consensus.cc:3096] T ebafa55443604f13824f211c1dcc6e41 P 42d9d1b29996412d9ff55347e3c665b3 [term 1 FOLLOWER]: Advancing to term 3
I20261115 21:02:28.791546 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 32306
I20261115 21:02:28.796715 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 32437
I20261115 21:02:28.801523 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 395
I20261115 21:02:28.806373 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 32699
I20261115 21:02:28.811244 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 32215
2026-11-15T21:02:28Z chronyd exiting
[ OK ] SlowTabletBootstrapTest.TestSlowBootstrap (8182 ms)
[ RUN ] SlowTabletBootstrapTest.TestFallBehind
I20261115 21:02:28.832613 30239 ts_itest-base.cc:115] Starting cluster with:
I20261115 21:02:28.832656 30239 ts_itest-base.cc:116] --------------
I20261115 21:02:28.832667 30239 ts_itest-base.cc:117] 4 tablet servers
I20261115 21:02:28.832677 30239 ts_itest-base.cc:118] 3 replicas per TS
I20261115 21:02:28.832686 30239 ts_itest-base.cc:119] --------------
2026-11-15T21:02:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-15T21:02:28Z Disabled control of system clock
I20261115 21:02:28.839982 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:41829
--webserver_interface=127.29.135.254
--webserver_port=0
--builtin_ntp_servers=127.29.135.212:40817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.135.254:41829 with env {}
W20261115 21:02:28.934187 564 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:28.934338 564 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:28.934393 564 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:28.937690 564 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261115 21:02:28.937747 564 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:28.937774 564 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261115 21:02:28.937799 564 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261115 21:02:28.941625 564 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:40817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.135.254:41829
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:41829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.29.135.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:28.942378 564 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:28.943094 564 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:28.947758 572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:28.947849 569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:28.947956 564 server_base.cc:1061] running on GCE node
W20261115 21:02:28.947860 570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:28.948500 564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:28.949157 564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:28.950299 564 hybrid_clock.cc:648] HybridClock initialized: now 1794776548950280 us; error 32 us; skew 500 ppm
I20261115 21:02:28.951792 564 webserver.cc:577] Webserver started at http://127.29.135.254:36535/ using document root <none> and password file <none>
I20261115 21:02:28.952316 564 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:28.952363 564 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:28.952574 564 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:28.953835 564 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "55c6813cc4db45848bec5ed502e1219c"
format_stamp: "Formatted at 2026-11-15 21:02:28 on dist-test-slave-xh31"
I20261115 21:02:28.954245 564 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "55c6813cc4db45848bec5ed502e1219c"
format_stamp: "Formatted at 2026-11-15 21:02:28 on dist-test-slave-xh31"
I20261115 21:02:28.957185 564 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20261115 21:02:28.959263 578 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:28.960206 564 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20261115 21:02:28.960331 564 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
uuid: "55c6813cc4db45848bec5ed502e1219c"
format_stamp: "Formatted at 2026-11-15 21:02:28 on dist-test-slave-xh31"
I20261115 21:02:28.960417 564 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:28.977272 564 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:28.977705 564 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:28.977849 564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:28.984257 630 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.254:41829 every 8 connection(s)
I20261115 21:02:28.984256 564 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.254:41829
I20261115 21:02:28.985219 564 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20261115 21:02:28.988133 631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:28.992791 631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c: Bootstrap starting.
I20261115 21:02:28.994110 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 564
I20261115 21:02:28.994221 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal/instance
I20261115 21:02:28.995180 631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:28.995996 631 log.cc:826] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:28.997435 631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c: No bootstrap required, opened a new log
I20261115 21:02:29.000219 631 raft_consensus.cc:360] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c6813cc4db45848bec5ed502e1219c" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 41829 } }
I20261115 21:02:29.000408 631 raft_consensus.cc:386] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:29.000475 631 raft_consensus.cc:748] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c6813cc4db45848bec5ed502e1219c, State: Initialized, Role: FOLLOWER
I20261115 21:02:29.000885 631 consensus_queue.cc:261] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c6813cc4db45848bec5ed502e1219c" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 41829 } }
I20261115 21:02:29.001017 631 raft_consensus.cc:400] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261115 21:02:29.001075 631 raft_consensus.cc:499] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261115 21:02:29.001153 631 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:29.001905 631 raft_consensus.cc:521] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c6813cc4db45848bec5ed502e1219c" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 41829 } }
I20261115 21:02:29.002210 631 leader_election.cc:304] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55c6813cc4db45848bec5ed502e1219c; no voters:
I20261115 21:02:29.002450 631 leader_election.cc:290] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [CANDIDATE]: Term 1 election: Requested vote from peers
I20261115 21:02:29.002547 636 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:29.002727 636 raft_consensus.cc:705] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [term 1 LEADER]: Becoming Leader. State: Replica: 55c6813cc4db45848bec5ed502e1219c, State: Running, Role: LEADER
I20261115 21:02:29.003046 636 consensus_queue.cc:238] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c6813cc4db45848bec5ed502e1219c" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 41829 } }
I20261115 21:02:29.003422 631 sys_catalog.cc:565] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [sys.catalog]: configured and running, proceeding with master startup.
I20261115 21:02:29.004438 638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55c6813cc4db45848bec5ed502e1219c. Latest consensus state: current_term: 1 leader_uuid: "55c6813cc4db45848bec5ed502e1219c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c6813cc4db45848bec5ed502e1219c" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 41829 } } }
I20261115 21:02:29.004546 638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:29.004890 645 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261115 21:02:29.005362 637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55c6813cc4db45848bec5ed502e1219c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c6813cc4db45848bec5ed502e1219c" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 41829 } } }
I20261115 21:02:29.005477 637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:29.007737 645 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261115 21:02:29.012332 645 catalog_manager.cc:1398] Generated new cluster ID: cd146724dd224b3ebfc4dc5f7d739860
I20261115 21:02:29.012432 645 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261115 21:02:29.039893 645 catalog_manager.cc:1421] Generated new certificate authority record
I20261115 21:02:29.040732 645 catalog_manager.cc:1555] Loading token signing keys...
I20261115 21:02:29.047133 645 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 55c6813cc4db45848bec5ed502e1219c: Generated new TSK 0
I20261115 21:02:29.047685 645 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261115 21:02:29.055300 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.193:0
--local_ip_for_outbound_sockets=127.29.135.193
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:41829
--builtin_ntp_servers=127.29.135.212:40817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261115 21:02:29.155277 655 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:29.155503 655 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:29.155541 655 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:29.155606 655 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:29.159003 655 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:29.159052 655 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:29.159102 655 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:29.159192 655 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:29.159229 655 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.193
I20261115 21:02:29.163151 655 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:40817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:41829
--never_fsync=true
--heap_profile_path=/tmp/kudu.655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:29.163904 655 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:29.164588 655 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:29.169771 660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:29.169800 661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:29.169826 663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:29.170042 655 server_base.cc:1061] running on GCE node
I20261115 21:02:29.170362 655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:29.170869 655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:29.172006 655 hybrid_clock.cc:648] HybridClock initialized: now 1794776549171985 us; error 38 us; skew 500 ppm
I20261115 21:02:29.173493 655 webserver.cc:577] Webserver started at http://127.29.135.193:33113/ using document root <none> and password file <none>
I20261115 21:02:29.174011 655 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:29.174060 655 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:29.174249 655 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:29.175503 655 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "210bcd36e63648928c4f6134861d2cf9"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.175915 655 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "210bcd36e63648928c4f6134861d2cf9"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.178999 655 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20261115 21:02:29.181131 669 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.182024 655 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:29.182123 655 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "210bcd36e63648928c4f6134861d2cf9"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.182200 655 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:29.196693 655 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:29.197125 655 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:29.197263 655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:29.197763 655 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:29.198724 655 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:29.198770 655 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.198827 655 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:29.198863 655 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.207167 655 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.193:36617
I20261115 21:02:29.207202 782 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.193:36617 every 8 connection(s)
I20261115 21:02:29.208089 655 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20261115 21:02:29.210640 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 655
I20261115 21:02:29.210747 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20261115 21:02:29.212782 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.194:0
--local_ip_for_outbound_sockets=127.29.135.194
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:41829
--builtin_ntp_servers=127.29.135.212:40817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261115 21:02:29.215642 783 heartbeater.cc:345] Connected to a master server at 127.29.135.254:41829
I20261115 21:02:29.215833 783 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:29.216282 783 heartbeater.cc:508] Master 127.29.135.254:41829 requested a full tablet report, sending...
I20261115 21:02:29.217734 595 ts_manager.cc:210] Registered new tserver with Master: 210bcd36e63648928c4f6134861d2cf9 (127.29.135.193:36617)
I20261115 21:02:29.218982 595 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.193:33105
W20261115 21:02:29.313406 786 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:29.313601 786 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:29.313642 786 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:29.313705 786 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:29.317028 786 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:29.317080 786 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:29.317126 786 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:29.317216 786 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:29.317252 786 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.194
I20261115 21:02:29.321085 786 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:40817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:41829
--never_fsync=true
--heap_profile_path=/tmp/kudu.786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:29.321784 786 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:29.322446 786 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:29.327504 792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:29.327505 791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:29.327557 794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:29.327741 786 server_base.cc:1061] running on GCE node
I20261115 21:02:29.328109 786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:29.328620 786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:29.329757 786 hybrid_clock.cc:648] HybridClock initialized: now 1794776549329745 us; error 24 us; skew 500 ppm
I20261115 21:02:29.331239 786 webserver.cc:577] Webserver started at http://127.29.135.194:38849/ using document root <none> and password file <none>
I20261115 21:02:29.331743 786 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:29.331790 786 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:29.331979 786 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:29.333187 786 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "5a99a3037dfe463ba9c97e22e8f8b422"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.333609 786 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "5a99a3037dfe463ba9c97e22e8f8b422"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.336579 786 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20261115 21:02:29.338625 800 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.339453 786 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:29.339550 786 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "5a99a3037dfe463ba9c97e22e8f8b422"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.339623 786 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:29.351572 786 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:29.351898 786 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:29.352017 786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:29.352414 786 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:29.353209 786 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:29.353253 786 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.353310 786 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:29.353340 786 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.361131 786 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.194:42477
I20261115 21:02:29.361155 913 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.194:42477 every 8 connection(s)
I20261115 21:02:29.361949 786 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261115 21:02:29.368300 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 786
I20261115 21:02:29.368350 914 heartbeater.cc:345] Connected to a master server at 127.29.135.254:41829
I20261115 21:02:29.368433 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20261115 21:02:29.368539 914 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:29.368911 914 heartbeater.cc:508] Master 127.29.135.254:41829 requested a full tablet report, sending...
I20261115 21:02:29.369874 594 ts_manager.cc:210] Registered new tserver with Master: 5a99a3037dfe463ba9c97e22e8f8b422 (127.29.135.194:42477)
I20261115 21:02:29.370577 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.195:0
--local_ip_for_outbound_sockets=127.29.135.195
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:41829
--builtin_ntp_servers=127.29.135.212:40817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261115 21:02:29.370615 594 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.194:46169
W20261115 21:02:29.466001 917 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:29.466189 917 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:29.466212 917 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:29.466272 917 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:29.469635 917 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:29.469689 917 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:29.469722 917 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:29.469805 917 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:29.469853 917 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.195
I20261115 21:02:29.473809 917 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:40817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:41829
--never_fsync=true
--heap_profile_path=/tmp/kudu.917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:29.474596 917 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:29.475263 917 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:29.480610 925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:29.480710 923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:29.480744 917 server_base.cc:1061] running on GCE node
W20261115 21:02:29.480610 922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:29.481251 917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:29.481791 917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:29.482951 917 hybrid_clock.cc:648] HybridClock initialized: now 1794776549482936 us; error 25 us; skew 500 ppm
I20261115 21:02:29.484547 917 webserver.cc:577] Webserver started at http://127.29.135.195:44481/ using document root <none> and password file <none>
I20261115 21:02:29.485083 917 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:29.485144 917 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:29.485337 917 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:29.486603 917 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "fc94764370e04ddfa596365e664d1563"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.486974 917 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "fc94764370e04ddfa596365e664d1563"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.489841 917 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20261115 21:02:29.491874 931 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.492724 917 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20261115 21:02:29.492827 917 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "fc94764370e04ddfa596365e664d1563"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.492908 917 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:29.504606 917 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:29.504973 917 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:29.505101 917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:29.505540 917 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:29.506398 917 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:29.506444 917 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.506532 917 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:29.506558 917 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.514689 917 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.195:37299
I20261115 21:02:29.514721 1045 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.195:37299 every 8 connection(s)
I20261115 21:02:29.515569 917 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261115 21:02:29.521865 1046 heartbeater.cc:345] Connected to a master server at 127.29.135.254:41829
I20261115 21:02:29.522044 1046 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:29.522420 1046 heartbeater.cc:508] Master 127.29.135.254:41829 requested a full tablet report, sending...
I20261115 21:02:29.523308 594 ts_manager.cc:210] Registered new tserver with Master: fc94764370e04ddfa596365e664d1563 (127.29.135.195:37299)
I20261115 21:02:29.523815 594 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.195:32845
I20261115 21:02:29.525223 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 917
I20261115 21:02:29.525322 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20261115 21:02:29.527218 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.196:0
--local_ip_for_outbound_sockets=127.29.135.196
--webserver_interface=127.29.135.196
--webserver_port=0
--tserver_master_addrs=127.29.135.254:41829
--builtin_ntp_servers=127.29.135.212:40817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261115 21:02:29.621116 1049 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:29.621280 1049 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:29.621300 1049 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:29.621343 1049 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:29.624560 1049 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:29.624603 1049 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:29.624636 1049 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:29.624692 1049 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:29.624719 1049 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.196
I20261115 21:02:29.628409 1049 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:40817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.29.135.196
--webserver_port=0
--tserver_master_addrs=127.29.135.254:41829
--never_fsync=true
--heap_profile_path=/tmp/kudu.1049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.196
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:29.628994 1049 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:29.629575 1049 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:29.634517 1057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:29.634595 1049 server_base.cc:1061] running on GCE node
W20261115 21:02:29.634610 1054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:29.634527 1055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:29.635070 1049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:29.635596 1049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:29.636746 1049 hybrid_clock.cc:648] HybridClock initialized: now 1794776549636724 us; error 36 us; skew 500 ppm
I20261115 21:02:29.638245 1049 webserver.cc:577] Webserver started at http://127.29.135.196:43965/ using document root <none> and password file <none>
I20261115 21:02:29.638785 1049 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:29.638845 1049 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:29.639056 1049 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:29.640239 1049 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "e9e236b799914396afa471ebb7dbd1a1"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.640657 1049 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "e9e236b799914396afa471ebb7dbd1a1"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.643715 1049 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20261115 21:02:29.645740 1063 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.646615 1049 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20261115 21:02:29.646713 1049 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "e9e236b799914396afa471ebb7dbd1a1"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:29.646792 1049 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:29.656180 1049 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:29.656539 1049 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:29.656672 1049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:29.657107 1049 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:29.657958 1049 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:29.658008 1049 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.658064 1049 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:29.658095 1049 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:29.667013 1049 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.196:41597
I20261115 21:02:29.667047 1176 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.196:41597 every 8 connection(s)
I20261115 21:02:29.667943 1049 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261115 21:02:29.671360 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 1049
I20261115 21:02:29.671499 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20261115 21:02:29.675355 1177 heartbeater.cc:345] Connected to a master server at 127.29.135.254:41829
I20261115 21:02:29.675540 1177 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:29.675946 1177 heartbeater.cc:508] Master 127.29.135.254:41829 requested a full tablet report, sending...
I20261115 21:02:29.676851 594 ts_manager.cc:210] Registered new tserver with Master: e9e236b799914396afa471ebb7dbd1a1 (127.29.135.196:41597)
I20261115 21:02:29.677357 594 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.196:49901
I20261115 21:02:29.685645 30239 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261115 21:02:29.693495 594 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40096:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20261115 21:02:29.711503 1111 tablet_service.cc:1500] Processing CreateTablet for tablet cb4c606fe9ac476fb7765d053fe02ac8 (DEFAULT_TABLE table=TestTable [id=1e55a3d5e0bd4f3898bd87fb74db9c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:29.711524 980 tablet_service.cc:1500] Processing CreateTablet for tablet cb4c606fe9ac476fb7765d053fe02ac8 (DEFAULT_TABLE table=TestTable [id=1e55a3d5e0bd4f3898bd87fb74db9c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:29.711524 848 tablet_service.cc:1500] Processing CreateTablet for tablet cb4c606fe9ac476fb7765d053fe02ac8 (DEFAULT_TABLE table=TestTable [id=1e55a3d5e0bd4f3898bd87fb74db9c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:29.712373 1111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb4c606fe9ac476fb7765d053fe02ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:29.712390 848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb4c606fe9ac476fb7765d053fe02ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:29.712390 980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb4c606fe9ac476fb7765d053fe02ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:29.717551 1194 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Bootstrap starting.
I20261115 21:02:29.718468 1195 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Bootstrap starting.
I20261115 21:02:29.718791 1196 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: Bootstrap starting.
I20261115 21:02:29.720012 1194 tablet_bootstrap.cc:654] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:29.720645 1195 tablet_bootstrap.cc:654] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:29.720666 1196 tablet_bootstrap.cc:654] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:29.720919 1194 log.cc:826] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:29.721383 1196 log.cc:826] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:29.721397 1195 log.cc:826] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:29.722332 1194 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: No bootstrap required, opened a new log
I20261115 21:02:29.722543 1194 ts_tablet_manager.cc:1449] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20261115 21:02:29.722697 1196 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: No bootstrap required, opened a new log
I20261115 21:02:29.722697 1195 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: No bootstrap required, opened a new log
I20261115 21:02:29.722884 1195 ts_tablet_manager.cc:1449] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20261115 21:02:29.722884 1196 ts_tablet_manager.cc:1449] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20261115 21:02:29.724938 1194 raft_consensus.cc:360] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:29.725237 1194 raft_consensus.cc:386] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:29.725297 1194 raft_consensus.cc:748] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e236b799914396afa471ebb7dbd1a1, State: Initialized, Role: FOLLOWER
I20261115 21:02:29.725700 1194 consensus_queue.cc:261] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:29.725865 1195 raft_consensus.cc:360] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:29.725865 1196 raft_consensus.cc:360] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:29.726053 1196 raft_consensus.cc:386] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:29.726128 1195 raft_consensus.cc:386] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:29.726136 1196 raft_consensus.cc:748] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc94764370e04ddfa596365e664d1563, State: Initialized, Role: FOLLOWER
I20261115 21:02:29.726329 1177 heartbeater.cc:500] Master 127.29.135.254:41829 was elected leader, sending a full tablet report...
I20261115 21:02:29.726449 1194 ts_tablet_manager.cc:1480] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20261115 21:02:29.726549 1196 consensus_queue.cc:261] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:29.727113 1046 heartbeater.cc:500] Master 127.29.135.254:41829 was elected leader, sending a full tablet report...
I20261115 21:02:29.727250 1196 ts_tablet_manager.cc:1480] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20261115 21:02:29.727494 1195 raft_consensus.cc:748] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a99a3037dfe463ba9c97e22e8f8b422, State: Initialized, Role: FOLLOWER
I20261115 21:02:29.727931 1195 consensus_queue.cc:261] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:29.728538 914 heartbeater.cc:500] Master 127.29.135.254:41829 was elected leader, sending a full tablet report...
I20261115 21:02:29.728668 1195 ts_tablet_manager.cc:1480] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20261115 21:02:29.732798 1200 raft_consensus.cc:499] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261115 21:02:29.732923 1200 raft_consensus.cc:521] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:29.733891 1200 leader_election.cc:290] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc94764370e04ddfa596365e664d1563 (127.29.135.195:37299), 5a99a3037dfe463ba9c97e22e8f8b422 (127.29.135.194:42477)
I20261115 21:02:29.737768 1000 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cb4c606fe9ac476fb7765d053fe02ac8" candidate_uuid: "e9e236b799914396afa471ebb7dbd1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc94764370e04ddfa596365e664d1563" is_pre_election: true
I20261115 21:02:29.737851 868 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cb4c606fe9ac476fb7765d053fe02ac8" candidate_uuid: "e9e236b799914396afa471ebb7dbd1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" is_pre_election: true
I20261115 21:02:29.738056 1000 raft_consensus.cc:2490] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9e236b799914396afa471ebb7dbd1a1 in term 0.
I20261115 21:02:29.738060 868 raft_consensus.cc:2490] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9e236b799914396afa471ebb7dbd1a1 in term 0.
I20261115 21:02:29.738472 1066 leader_election.cc:304] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a99a3037dfe463ba9c97e22e8f8b422, e9e236b799914396afa471ebb7dbd1a1; no voters:
I20261115 21:02:29.738744 1200 raft_consensus.cc:2826] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261115 21:02:29.738803 1200 raft_consensus.cc:499] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261115 21:02:29.738849 1200 raft_consensus.cc:3096] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:29.739698 1200 raft_consensus.cc:521] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:29.740039 1200 leader_election.cc:290] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [CANDIDATE]: Term 1 election: Requested vote from peers fc94764370e04ddfa596365e664d1563 (127.29.135.195:37299), 5a99a3037dfe463ba9c97e22e8f8b422 (127.29.135.194:42477)
I20261115 21:02:29.740365 868 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cb4c606fe9ac476fb7765d053fe02ac8" candidate_uuid: "e9e236b799914396afa471ebb7dbd1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a99a3037dfe463ba9c97e22e8f8b422"
I20261115 21:02:29.740502 868 raft_consensus.cc:3096] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:29.741302 868 raft_consensus.cc:2490] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9e236b799914396afa471ebb7dbd1a1 in term 1.
I20261115 21:02:29.741516 1000 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cb4c606fe9ac476fb7765d053fe02ac8" candidate_uuid: "e9e236b799914396afa471ebb7dbd1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc94764370e04ddfa596365e664d1563"
I20261115 21:02:29.741544 1066 leader_election.cc:304] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a99a3037dfe463ba9c97e22e8f8b422, e9e236b799914396afa471ebb7dbd1a1; no voters:
I20261115 21:02:29.741616 1000 raft_consensus.cc:3096] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:29.741679 1200 raft_consensus.cc:2826] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:29.741851 1200 raft_consensus.cc:705] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 1 LEADER]: Becoming Leader. State: Replica: e9e236b799914396afa471ebb7dbd1a1, State: Running, Role: LEADER
I20261115 21:02:29.742138 1200 consensus_queue.cc:238] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:29.742538 1000 raft_consensus.cc:2490] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9e236b799914396afa471ebb7dbd1a1 in term 1.
I20261115 21:02:29.745396 1204 consensus_queue.cc:1049] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:29.745537 1200 consensus_queue.cc:1049] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:29.745859 595 catalog_manager.cc:5840] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9e236b799914396afa471ebb7dbd1a1 (127.29.135.196). New cstate: current_term: 1 leader_uuid: "e9e236b799914396afa471ebb7dbd1a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } health_report { overall_health: UNKNOWN } } }
I20261115 21:02:29.753800 30239 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261115 21:02:29.755095 30239 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 5a99a3037dfe463ba9c97e22e8f8b422 to finish bootstrapping
I20261115 21:02:29.760524 30239 ts_itest-base.cc:246] Waiting for 1 tablets on tserver fc94764370e04ddfa596365e664d1563 to finish bootstrapping
I20261115 21:02:29.765244 30239 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e9e236b799914396afa471ebb7dbd1a1 to finish bootstrapping
I20261115 21:02:29.769629 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 786
I20261115 21:02:29.791273 30239 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20261115 21:02:29.791561 30239 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
W20261115 21:02:29.806241 1066 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.135.194:42477: connect: Connection refused (error 111)
W20261115 21:02:29.807137 1066 consensus_peers.cc:597] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 -> Peer 5a99a3037dfe463ba9c97e22e8f8b422 (127.29.135.194:42477): Couldn't send request to peer 5a99a3037dfe463ba9c97e22e8f8b422. Status: Network error: Client connection negotiation failed: client connection to 127.29.135.194:42477: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261115 21:02:29.870186 1178 maintenance_manager.cc:419] P e9e236b799914396afa471ebb7dbd1a1: Scheduling FlushMRSOp(cb4c606fe9ac476fb7765d053fe02ac8): 3681784 bytes log retention, and flush 5079305 bytes memory
I20261115 21:02:29.892987 30239 raft_consensus-itest-base.cc:274] Waiting for log GC on e9e236b799914396afa471ebb7dbd1a1
I20261115 21:02:29.917636 1047 maintenance_manager.cc:419] P fc94764370e04ddfa596365e664d1563: Scheduling FlushMRSOp(cb4c606fe9ac476fb7765d053fe02ac8): 4470599 bytes log retention, and flush 6119689 bytes memory
I20261115 21:02:29.935093 1068 maintenance_manager.cc:643] P e9e236b799914396afa471ebb7dbd1a1: FlushMRSOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.065s user 0.052s sys 0.000s Metrics: {"bytes_written":135937,"cfile_init":1,"compiler_manager_pool.queue_time_us":196,"dirs.queue_time_us":46,"dirs.run_cpu_time_us":159,"dirs.run_wall_time_us":843,"drs_written":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4,"lbm_write_time_us":597,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":27,"thread_start_us":108,"threads_started":1}
I20261115 21:02:29.936196 1178 maintenance_manager.cc:419] P e9e236b799914396afa471ebb7dbd1a1: Scheduling LogGCOp(cb4c606fe9ac476fb7765d053fe02ac8): free 2761328 bytes of WAL
I20261115 21:02:29.936506 1068 log_reader.cc:385] T cb4c606fe9ac476fb7765d053fe02ac8: removed 3 log segments from log reader
I20261115 21:02:29.936612 1068 log.cc:1079] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb4c606fe9ac476fb7765d053fe02ac8/wal-000000001 (ops 1-8)
I20261115 21:02:29.936733 1068 log.cc:1079] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb4c606fe9ac476fb7765d053fe02ac8/wal-000000002 (ops 9-15)
I20261115 21:02:29.936811 1068 log.cc:1079] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb4c606fe9ac476fb7765d053fe02ac8/wal-000000003 (ops 16-22)
I20261115 21:02:29.937816 1068 maintenance_manager.cc:643] P e9e236b799914396afa471ebb7dbd1a1: LogGCOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261115 21:02:29.938167 1178 maintenance_manager.cc:419] P e9e236b799914396afa471ebb7dbd1a1: Scheduling FlushMRSOp(cb4c606fe9ac476fb7765d053fe02ac8): 1840874 bytes log retention, and flush 1958153 bytes memory
I20261115 21:02:29.944118 30239 raft_consensus-itest-base.cc:279] Log GC complete on e9e236b799914396afa471ebb7dbd1a1
I20261115 21:02:29.955366 1068 maintenance_manager.cc:643] P e9e236b799914396afa471ebb7dbd1a1: FlushMRSOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.017s user 0.015s sys 0.001s Metrics: {"bytes_written":135803,"cfile_init":1,"dirs.queue_time_us":40,"dirs.run_cpu_time_us":150,"dirs.run_wall_time_us":1182,"drs_written":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4,"lbm_write_time_us":621,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":9}
I20261115 21:02:29.956408 1178 maintenance_manager.cc:419] P e9e236b799914396afa471ebb7dbd1a1: Scheduling CompactRowSetsOp(cb4c606fe9ac476fb7765d053fe02ac8): perf score=0.963683
I20261115 21:02:30.013859 1068 maintenance_manager.cc:643] P e9e236b799914396afa471ebb7dbd1a1: CompactRowSetsOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.057s user 0.054s sys 0.002s Metrics: {"bytes_written":136001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262393,"cfile_cache_miss":26,"cfile_cache_miss_bytes":264439,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":88,"dirs.run_cpu_time_us":165,"dirs.run_wall_time_us":1275,"drs_written":1,"lbm_read_time_us":732,"lbm_reads_lt_1ms":54,"lbm_write_time_us":624,"lbm_writes_lt_1ms":23,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":36,"thread_start_us":317,"threads_started":5}
I20261115 21:02:30.014734 1178 maintenance_manager.cc:419] P e9e236b799914396afa471ebb7dbd1a1: Scheduling UndoDeltaBlockGCOp(cb4c606fe9ac476fb7765d053fe02ac8): 536 bytes on disk
I20261115 21:02:30.015527 1068 maintenance_manager.cc:643] P e9e236b799914396afa471ebb7dbd1a1: UndoDeltaBlockGCOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4}
I20261115 21:02:30.025377 936 maintenance_manager.cc:643] P fc94764370e04ddfa596365e664d1563: FlushMRSOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.107s user 0.056s sys 0.004s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":14876,"drs_written":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4,"lbm_write_time_us":634,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":36,"thread_start_us":175,"threads_started":2}
I20261115 21:02:30.026270 1047 maintenance_manager.cc:419] P fc94764370e04ddfa596365e664d1563: Scheduling LogGCOp(cb4c606fe9ac476fb7765d053fe02ac8): free 2761276 bytes of WAL
I20261115 21:02:30.026574 936 log_reader.cc:385] T cb4c606fe9ac476fb7765d053fe02ac8: removed 3 log segments from log reader
I20261115 21:02:30.026674 936 log.cc:1079] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb4c606fe9ac476fb7765d053fe02ac8/wal-000000001 (ops 1-8)
I20261115 21:02:30.026782 936 log.cc:1079] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb4c606fe9ac476fb7765d053fe02ac8/wal-000000002 (ops 9-15)
I20261115 21:02:30.026851 936 log.cc:1079] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb4c606fe9ac476fb7765d053fe02ac8/wal-000000003 (ops 16-22)
I20261115 21:02:30.027765 936 maintenance_manager.cc:643] P fc94764370e04ddfa596365e664d1563: LogGCOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261115 21:02:30.028025 1047 maintenance_manager.cc:419] P fc94764370e04ddfa596365e664d1563: Scheduling UndoDeltaBlockGCOp(cb4c606fe9ac476fb7765d053fe02ac8): 536 bytes on disk
I20261115 21:02:30.028656 936 maintenance_manager.cc:643] P fc94764370e04ddfa596365e664d1563: UndoDeltaBlockGCOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4}
I20261115 21:02:30.220893 783 heartbeater.cc:500] Master 127.29.135.254:41829 was elected leader, sending a full tablet report...
I20261115 21:02:30.307950 1230 consensus_queue.cc:754] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: The logs necessary to catch up peer 5a99a3037dfe463ba9c97e22e8f8b422 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 3..31: Segment 1 which contained index 3 has been GCed)
I20261115 21:02:30.308054 1230 consensus_queue.cc:580] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: The logs necessary to catch up peer 5a99a3037dfe463ba9c97e22e8f8b422 have been garbage collected. The replica will never be able to catch up
I20261115 21:02:30.310986 1131 consensus_queue.cc:238] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 1.37, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } }
I20261115 21:02:30.312078 999 raft_consensus.cc:1285] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 1 FOLLOWER]: Refusing update from remote peer e9e236b799914396afa471ebb7dbd1a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 37. Preceding OpId from leader: term: 1 index: 38. (index mismatch)
I20261115 21:02:30.312443 1200 consensus_queue.cc:1049] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20261115 21:02:30.313903 1230 raft_consensus.cc:2983] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 1 LEADER]: Committing config change with OpId 1.38: config changed from index -1 to 38, VOTER 5a99a3037dfe463ba9c97e22e8f8b422 (127.29.135.194) evicted. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } }
I20261115 21:02:30.314262 999 raft_consensus.cc:2983] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 1 FOLLOWER]: Committing config change with OpId 1.38: config changed from index -1 to 38, VOTER 5a99a3037dfe463ba9c97e22e8f8b422 (127.29.135.194) evicted. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } }
I20261115 21:02:30.315732 582 catalog_manager.cc:5247] ChangeConfig:REMOVE_PEER RPC for tablet cb4c606fe9ac476fb7765d053fe02ac8 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20261115 21:02:30.316548 593 catalog_manager.cc:5840] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 reported cstate change: config changed from index -1 to 38, VOTER 5a99a3037dfe463ba9c97e22e8f8b422 (127.29.135.194) evicted. New cstate: current_term: 1 leader_uuid: "e9e236b799914396afa471ebb7dbd1a1" committed_config { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } health_report { overall_health: HEALTHY } } }
I20261115 21:02:30.316919 1178 maintenance_manager.cc:419] P e9e236b799914396afa471ebb7dbd1a1: Scheduling LogGCOp(cb4c606fe9ac476fb7765d053fe02ac8): free 1840874 bytes of WAL
I20261115 21:02:30.317118 1068 log_reader.cc:385] T cb4c606fe9ac476fb7765d053fe02ac8: removed 2 log segments from log reader
I20261115 21:02:30.317174 1068 log.cc:1079] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb4c606fe9ac476fb7765d053fe02ac8/wal-000000004 (ops 23-29)
I20261115 21:02:30.317217 1068 log.cc:1079] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb4c606fe9ac476fb7765d053fe02ac8/wal-000000005 (ops 30-36)
I20261115 21:02:30.317698 1068 maintenance_manager.cc:643] P e9e236b799914396afa471ebb7dbd1a1: LogGCOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
W20261115 21:02:30.319496 581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.135.194:42477: connect: Connection refused (error 111)
I20261115 21:02:30.320092 1131 consensus_queue.cc:238] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 1.38, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: NON_VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: true } }
W20261115 21:02:30.320287 581 catalog_manager.cc:4792] TS 5a99a3037dfe463ba9c97e22e8f8b422 (127.29.135.194:42477): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cb4c606fe9ac476fb7765d053fe02ac8: Network error: Client connection negotiation failed: client connection to 127.29.135.194:42477: connect: Connection refused (error 111)
I20261115 21:02:30.321305 999 raft_consensus.cc:1285] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 1 FOLLOWER]: Refusing update from remote peer e9e236b799914396afa471ebb7dbd1a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 38. Preceding OpId from leader: term: 1 index: 39. (index mismatch)
I20261115 21:02:30.321635 1200 consensus_queue.cc:1049] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20261115 21:02:30.323030 1230 raft_consensus.cc:2983] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 1 LEADER]: Committing config change with OpId 1.39: config changed from index 38 to 39, NON_VOTER 210bcd36e63648928c4f6134861d2cf9 (127.29.135.193) added. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: NON_VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: true } } }
I20261115 21:02:30.324236 582 catalog_manager.cc:5247] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet cb4c606fe9ac476fb7765d053fe02ac8 with cas_config_opid_index 38: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20261115 21:02:30.324635 999 raft_consensus.cc:2983] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 1 FOLLOWER]: Committing config change with OpId 1.39: config changed from index 38 to 39, NON_VOTER 210bcd36e63648928c4f6134861d2cf9 (127.29.135.193) added. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: NON_VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: true } } }
I20261115 21:02:30.324870 593 catalog_manager.cc:5840] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 reported cstate change: config changed from index 38 to 39, NON_VOTER 210bcd36e63648928c4f6134861d2cf9 (127.29.135.193) added. New cstate: current_term: 1 leader_uuid: "e9e236b799914396afa471ebb7dbd1a1" committed_config { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: NON_VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20261115 21:02:30.326576 1065 consensus_peers.cc:597] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 -> Peer 210bcd36e63648928c4f6134861d2cf9 (127.29.135.193:36617): Couldn't send request to peer 210bcd36e63648928c4f6134861d2cf9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb4c606fe9ac476fb7765d053fe02ac8. This is attempt 1: this message will repeat every 5th retry.
I20261115 21:02:30.831537 1250 ts_tablet_manager.cc:979] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: Initiating tablet copy from peer e9e236b799914396afa471ebb7dbd1a1 (127.29.135.196:41597)
I20261115 21:02:30.832055 1250 tablet_copy_client.cc:323] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: tablet copy: Beginning tablet copy session from remote peer at address 127.29.135.196:41597
I20261115 21:02:30.835331 1151 tablet_copy_service.cc:140] P e9e236b799914396afa471ebb7dbd1a1: Received BeginTabletCopySession request for tablet cb4c606fe9ac476fb7765d053fe02ac8 from peer 210bcd36e63648928c4f6134861d2cf9 ({username='slave'} at 127.29.135.193:56161)
I20261115 21:02:30.835454 1151 tablet_copy_service.cc:161] P e9e236b799914396afa471ebb7dbd1a1: Beginning new tablet copy session on tablet cb4c606fe9ac476fb7765d053fe02ac8 from peer 210bcd36e63648928c4f6134861d2cf9 at {username='slave'} at 127.29.135.193:56161: session id = 210bcd36e63648928c4f6134861d2cf9-cb4c606fe9ac476fb7765d053fe02ac8
I20261115 21:02:30.836841 1151 tablet_copy_source_session.cc:215] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: Tablet Copy: opened 5 blocks and 1 log segments
I20261115 21:02:30.840215 1250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb4c606fe9ac476fb7765d053fe02ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:30.844575 1250 tablet_copy_client.cc:806] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: tablet copy: Starting download of 5 data blocks...
I20261115 21:02:30.850544 1250 tablet_copy_client.cc:670] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: tablet copy: Starting download of 1 WAL segments...
I20261115 21:02:30.852408 1250 tablet_copy_client.cc:538] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261115 21:02:30.854112 1250 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: Bootstrap starting.
I20261115 21:02:30.860805 1250 log.cc:826] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:30.864565 1250 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=1} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=3. Pending: 0 replicates
I20261115 21:02:30.864801 1250 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: Bootstrap complete.
I20261115 21:02:30.865057 1250 ts_tablet_manager.cc:1449] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.001s
I20261115 21:02:30.866680 1250 raft_consensus.cc:360] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: NON_VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: true } }
I20261115 21:02:30.866851 1250 raft_consensus.cc:748] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 210bcd36e63648928c4f6134861d2cf9, State: Initialized, Role: LEARNER
I20261115 21:02:30.867151 1250 consensus_queue.cc:261] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 39, Last appended: 1.39, Last appended by leader: 39, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: NON_VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: true } }
I20261115 21:02:30.867748 1250 ts_tablet_manager.cc:1480] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20261115 21:02:30.868275 1151 tablet_copy_service.cc:342] P e9e236b799914396afa471ebb7dbd1a1: Request end of tablet copy session 210bcd36e63648928c4f6134861d2cf9-cb4c606fe9ac476fb7765d053fe02ac8 received from {username='slave'} at 127.29.135.193:56161
I20261115 21:02:30.868352 1151 tablet_copy_service.cc:434] P e9e236b799914396afa471ebb7dbd1a1: ending tablet copy session 210bcd36e63648928c4f6134861d2cf9-cb4c606fe9ac476fb7765d053fe02ac8 on tablet cb4c606fe9ac476fb7765d053fe02ac8 with peer 210bcd36e63648928c4f6134861d2cf9
I20261115 21:02:30.913064 784 maintenance_manager.cc:419] P 210bcd36e63648928c4f6134861d2cf9: Scheduling UndoDeltaBlockGCOp(cb4c606fe9ac476fb7765d053fe02ac8): 536 bytes on disk
I20261115 21:02:30.913708 674 maintenance_manager.cc:643] P 210bcd36e63648928c4f6134861d2cf9: UndoDeltaBlockGCOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":39,"lbm_reads_lt_1ms":4}
I20261115 21:02:31.244998 737 raft_consensus.cc:1227] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.38->[1.39-1.39] Dedup: 1.39->[]
I20261115 21:02:31.433097 1047 maintenance_manager.cc:419] P fc94764370e04ddfa596365e664d1563: Scheduling LogGCOp(cb4c606fe9ac476fb7765d053fe02ac8): free 1709323 bytes of WAL
I20261115 21:02:31.433373 936 log_reader.cc:385] T cb4c606fe9ac476fb7765d053fe02ac8: removed 2 log segments from log reader
I20261115 21:02:31.433418 936 log.cc:1079] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb4c606fe9ac476fb7765d053fe02ac8/wal-000000004 (ops 23-28)
I20261115 21:02:31.433450 936 log.cc:1079] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb4c606fe9ac476fb7765d053fe02ac8/wal-000000005 (ops 29-35)
I20261115 21:02:31.433894 936 maintenance_manager.cc:643] P fc94764370e04ddfa596365e664d1563: LogGCOp(cb4c606fe9ac476fb7765d053fe02ac8) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
W20261115 21:02:31.529155 581 catalog_manager.cc:4792] TS 5a99a3037dfe463ba9c97e22e8f8b422 (127.29.135.194:42477): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cb4c606fe9ac476fb7765d053fe02ac8: Network error: Client connection negotiation failed: client connection to 127.29.135.194:42477: connect: Connection refused (error 111)
I20261115 21:02:31.685672 1249 raft_consensus.cc:1074] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1: attempting to promote NON_VOTER 210bcd36e63648928c4f6134861d2cf9 to VOTER
I20261115 21:02:31.686127 1249 consensus_queue.cc:238] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 1.39, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: false } }
I20261115 21:02:31.687364 999 raft_consensus.cc:1285] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 1 FOLLOWER]: Refusing update from remote peer e9e236b799914396afa471ebb7dbd1a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 40. (index mismatch)
I20261115 21:02:31.687389 737 raft_consensus.cc:1285] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9 [term 1 LEARNER]: Refusing update from remote peer e9e236b799914396afa471ebb7dbd1a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 40. (index mismatch)
I20261115 21:02:31.687789 1256 consensus_queue.cc:1049] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20261115 21:02:31.688004 1249 consensus_queue.cc:1049] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20261115 21:02:31.689564 1256 raft_consensus.cc:2983] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 [term 1 LEADER]: Committing config change with OpId 1.40: config changed from index 39 to 40, 210bcd36e63648928c4f6134861d2cf9 (127.29.135.193) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: false } } }
I20261115 21:02:31.689930 999 raft_consensus.cc:2983] T cb4c606fe9ac476fb7765d053fe02ac8 P fc94764370e04ddfa596365e664d1563 [term 1 FOLLOWER]: Committing config change with OpId 1.40: config changed from index 39 to 40, 210bcd36e63648928c4f6134861d2cf9 (127.29.135.193) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: false } } }
I20261115 21:02:31.691131 737 raft_consensus.cc:2983] T cb4c606fe9ac476fb7765d053fe02ac8 P 210bcd36e63648928c4f6134861d2cf9 [term 1 FOLLOWER]: Committing config change with OpId 1.40: config changed from index 39 to 40, 210bcd36e63648928c4f6134861d2cf9 (127.29.135.193) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: false } } }
I20261115 21:02:31.692158 594 catalog_manager.cc:5840] T cb4c606fe9ac476fb7765d053fe02ac8 P e9e236b799914396afa471ebb7dbd1a1 reported cstate change: config changed from index 39 to 40, 210bcd36e63648928c4f6134861d2cf9 (127.29.135.193) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9e236b799914396afa471ebb7dbd1a1" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210bcd36e63648928c4f6134861d2cf9" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 36617 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20261115 21:02:31.945320 30239 raft_consensus-itest-base.cc:295] Servers converged with original term 1
I20261115 21:02:31.945387 30239 raft_consensus-itest-base.cc:303] Restarting 5a99a3037dfe463ba9c97e22e8f8b422
I20261115 21:02:31.945691 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.194:42477
--local_ip_for_outbound_sockets=127.29.135.194
--tserver_master_addrs=127.29.135.254:41829
--webserver_port=38849
--webserver_interface=127.29.135.194
--builtin_ntp_servers=127.29.135.212:40817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261115 21:02:32.041854 1267 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:32.042032 1267 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:32.042052 1267 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:32.042099 1267 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:32.045293 1267 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:32.045337 1267 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:32.045369 1267 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:32.045428 1267 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:32.045455 1267 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.194
I20261115 21:02:32.049178 1267 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:40817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.194:42477
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.29.135.194
--webserver_port=38849
--tserver_master_addrs=127.29.135.254:41829
--never_fsync=true
--heap_profile_path=/tmp/kudu.1267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:32.049845 1267 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:32.050534 1267 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:32.055627 1272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:32.055675 1273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:32.055702 1275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:32.055868 1267 server_base.cc:1061] running on GCE node
I20261115 21:02:32.056192 1267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:32.056722 1267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:32.057888 1267 hybrid_clock.cc:648] HybridClock initialized: now 1794776552057873 us; error 23 us; skew 500 ppm
I20261115 21:02:32.059473 1267 webserver.cc:577] Webserver started at http://127.29.135.194:38849/ using document root <none> and password file <none>
I20261115 21:02:32.060012 1267 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:32.060088 1267 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:32.063249 1267 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20261115 21:02:32.065168 1281 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.066017 1267 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20261115 21:02:32.066120 1267 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "5a99a3037dfe463ba9c97e22e8f8b422"
format_stamp: "Formatted at 2026-11-15 21:02:29 on dist-test-slave-xh31"
I20261115 21:02:32.066479 1267 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:32.074663 1267 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:32.075019 1267 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:32.075155 1267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:32.075587 1267 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:32.076633 1288 ts_tablet_manager.cc:588] Loading tablet metadata (0/1 complete)
I20261115 21:02:32.079748 1267 ts_tablet_manager.cc:631] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20261115 21:02:32.079814 1267 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20261115 21:02:32.079876 1267 ts_tablet_manager.cc:646] Registering tablets (0/1 complete)
I20261115 21:02:32.081331 1267 ts_tablet_manager.cc:662] Registered 1 tablets
I20261115 21:02:32.081372 1267 ts_tablet_manager.cc:641] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20261115 21:02:32.081688 1288 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Bootstrap starting.
I20261115 21:02:32.089454 1288 log.cc:826] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:32.090173 1267 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.194:42477
I20261115 21:02:32.090238 1396 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.194:42477 every 8 connection(s)
I20261115 21:02:32.091105 1267 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261115 21:02:32.091770 1288 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261115 21:02:32.092072 1288 tablet_bootstrap.cc:492] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Bootstrap complete.
I20261115 21:02:32.092406 1288 ts_tablet_manager.cc:1449] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20261115 21:02:32.095592 1288 raft_consensus.cc:360] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:32.095878 1288 raft_consensus.cc:748] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a99a3037dfe463ba9c97e22e8f8b422, State: Initialized, Role: FOLLOWER
I20261115 21:02:32.096272 1288 consensus_queue.cc:261] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc94764370e04ddfa596365e664d1563" member_type: VOTER last_known_addr { host: "127.29.135.195" port: 37299 } } peers { permanent_uuid: "e9e236b799914396afa471ebb7dbd1a1" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 41597 } } peers { permanent_uuid: "5a99a3037dfe463ba9c97e22e8f8b422" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 42477 } }
I20261115 21:02:32.096995 1288 ts_tablet_manager.cc:1480] T cb4c606fe9ac476fb7765d053fe02ac8 P 5a99a3037dfe463ba9c97e22e8f8b422: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20261115 21:02:32.098618 1397 heartbeater.cc:345] Connected to a master server at 127.29.135.254:41829
I20261115 21:02:32.098817 1397 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:32.099210 1397 heartbeater.cc:508] Master 127.29.135.254:41829 requested a full tablet report, sending...
I20261115 21:02:32.100648 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 1267
I20261115 21:02:32.100903 593 ts_manager.cc:210] Re-registered known tserver with Master: 5a99a3037dfe463ba9c97e22e8f8b422 (127.29.135.194:42477)
I20261115 21:02:32.101815 593 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.194:53809
I20261115 21:02:32.103451 1397 heartbeater.cc:500] Master 127.29.135.254:41829 was elected leader, sending a full tablet report...
I20261115 21:02:32.104035 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 655
I20261115 21:02:32.110846 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 1267
I20261115 21:02:32.115736 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 917
I20261115 21:02:32.121748 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 1049
I20261115 21:02:32.127962 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 564
2026-11-15T21:02:32Z chronyd exiting
[ OK ] SlowTabletBootstrapTest.TestFallBehind (3317 ms)
[ RUN ] SlowTabletBootstrapTest.TestFallBehindSlowBootstrap
I20261115 21:02:32.150169 30239 ts_itest-base.cc:115] Starting cluster with:
I20261115 21:02:32.150211 30239 ts_itest-base.cc:116] --------------
I20261115 21:02:32.150221 30239 ts_itest-base.cc:117] 4 tablet servers
I20261115 21:02:32.150231 30239 ts_itest-base.cc:118] 3 replicas per TS
I20261115 21:02:32.150243 30239 ts_itest-base.cc:119] --------------
2026-11-15T21:02:32Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-15T21:02:32Z Disabled control of system clock
I20261115 21:02:32.157639 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:33623
--webserver_interface=127.29.135.254
--webserver_port=0
--builtin_ntp_servers=127.29.135.212:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.135.254:33623 with env {}
W20261115 21:02:32.251386 1409 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:32.251531 1409 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:32.251583 1409 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:32.254812 1409 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261115 21:02:32.254860 1409 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:32.254889 1409 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261115 21:02:32.254918 1409 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261115 21:02:32.258625 1409 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.135.254:33623
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.135.254:33623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.29.135.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.1409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:32.259274 1409 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:32.259891 1409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:32.264241 1415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:32.264285 1417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:32.264238 1414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:32.264417 1409 server_base.cc:1061] running on GCE node
I20261115 21:02:32.264798 1409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:32.265411 1409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:32.266542 1409 hybrid_clock.cc:648] HybridClock initialized: now 1794776552266519 us; error 35 us; skew 500 ppm
I20261115 21:02:32.267884 1409 webserver.cc:577] Webserver started at http://127.29.135.254:36593/ using document root <none> and password file <none>
I20261115 21:02:32.268363 1409 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:32.268409 1409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:32.268595 1409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:32.269788 1409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "0239fcf47aed41148a236bf3124c6ffe"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.270164 1409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "0239fcf47aed41148a236bf3124c6ffe"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.272969 1409 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20261115 21:02:32.274878 1423 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.275683 1409 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:32.275790 1409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
uuid: "0239fcf47aed41148a236bf3124c6ffe"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.275869 1409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:32.296015 1409 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:32.296406 1409 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:32.296527 1409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:32.302966 1409 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.254:33623
I20261115 21:02:32.302968 1475 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.254:33623 every 8 connection(s)
I20261115 21:02:32.303854 1409 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20261115 21:02:32.306658 1476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:32.311475 1476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe: Bootstrap starting.
I20261115 21:02:32.311682 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 1409
I20261115 21:02:32.311805 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/master-0/wal/instance
I20261115 21:02:32.313934 1476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:32.314851 1476 log.cc:826] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:32.316210 1476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe: No bootstrap required, opened a new log
I20261115 21:02:32.319139 1476 raft_consensus.cc:360] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0239fcf47aed41148a236bf3124c6ffe" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 33623 } }
I20261115 21:02:32.319340 1476 raft_consensus.cc:386] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:32.319406 1476 raft_consensus.cc:748] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0239fcf47aed41148a236bf3124c6ffe, State: Initialized, Role: FOLLOWER
I20261115 21:02:32.319756 1476 consensus_queue.cc:261] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0239fcf47aed41148a236bf3124c6ffe" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 33623 } }
I20261115 21:02:32.319880 1476 raft_consensus.cc:400] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261115 21:02:32.319940 1476 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261115 21:02:32.320024 1476 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:32.320798 1476 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0239fcf47aed41148a236bf3124c6ffe" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 33623 } }
I20261115 21:02:32.321125 1476 leader_election.cc:304] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0239fcf47aed41148a236bf3124c6ffe; no voters:
I20261115 21:02:32.321376 1476 leader_election.cc:290] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [CANDIDATE]: Term 1 election: Requested vote from peers
I20261115 21:02:32.321460 1481 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:32.321627 1481 raft_consensus.cc:705] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [term 1 LEADER]: Becoming Leader. State: Replica: 0239fcf47aed41148a236bf3124c6ffe, State: Running, Role: LEADER
I20261115 21:02:32.321962 1481 consensus_queue.cc:238] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0239fcf47aed41148a236bf3124c6ffe" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 33623 } }
I20261115 21:02:32.322329 1476 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [sys.catalog]: configured and running, proceeding with master startup.
I20261115 21:02:32.323362 1482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0239fcf47aed41148a236bf3124c6ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0239fcf47aed41148a236bf3124c6ffe" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 33623 } } }
I20261115 21:02:32.323460 1482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:32.323344 1483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0239fcf47aed41148a236bf3124c6ffe. Latest consensus state: current_term: 1 leader_uuid: "0239fcf47aed41148a236bf3124c6ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0239fcf47aed41148a236bf3124c6ffe" member_type: VOTER last_known_addr { host: "127.29.135.254" port: 33623 } } }
I20261115 21:02:32.323988 1483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe [sys.catalog]: This master's current role is: LEADER
I20261115 21:02:32.324615 1490 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261115 21:02:32.327095 1490 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261115 21:02:32.331460 1490 catalog_manager.cc:1398] Generated new cluster ID: c4b0830581a0425480562c4f41722493
I20261115 21:02:32.331518 1490 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261115 21:02:32.344403 1490 catalog_manager.cc:1421] Generated new certificate authority record
I20261115 21:02:32.345391 1490 catalog_manager.cc:1555] Loading token signing keys...
I20261115 21:02:32.352855 1490 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 0239fcf47aed41148a236bf3124c6ffe: Generated new TSK 0
I20261115 21:02:32.353416 1490 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261115 21:02:32.359527 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.193:0
--local_ip_for_outbound_sockets=127.29.135.193
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:33623
--builtin_ntp_servers=127.29.135.212:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261115 21:02:32.455847 1500 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:32.456022 1500 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:32.456043 1500 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:32.456095 1500 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:32.459400 1500 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:32.459446 1500 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:32.459481 1500 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:32.459559 1500 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:32.459599 1500 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.193
I20261115 21:02:32.463562 1500 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.29.135.193
--webserver_port=0
--tserver_master_addrs=127.29.135.254:33623
--never_fsync=true
--heap_profile_path=/tmp/kudu.1500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:32.464268 1500 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:32.464934 1500 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:32.471037 1505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:32.471076 1508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:32.471055 1506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:32.471311 1500 server_base.cc:1061] running on GCE node
I20261115 21:02:32.471653 1500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:32.472198 1500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:32.473397 1500 hybrid_clock.cc:648] HybridClock initialized: now 1794776552473372 us; error 44 us; skew 500 ppm
I20261115 21:02:32.475069 1500 webserver.cc:577] Webserver started at http://127.29.135.193:38347/ using document root <none> and password file <none>
I20261115 21:02:32.475615 1500 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:32.475660 1500 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:32.475871 1500 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:32.477137 1500 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "f21762a3da024f72b68d0a6abebfcc2d"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.477545 1500 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "f21762a3da024f72b68d0a6abebfcc2d"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.480675 1500 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20261115 21:02:32.482813 1514 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.483728 1500 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:32.483822 1500 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "f21762a3da024f72b68d0a6abebfcc2d"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.483906 1500 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:32.498697 1500 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:32.499073 1500 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:32.499195 1500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:32.499652 1500 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:32.500531 1500 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:32.500576 1500 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.500622 1500 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:32.500648 1500 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.508806 1500 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.193:40275
I20261115 21:02:32.508859 1627 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.193:40275 every 8 connection(s)
I20261115 21:02:32.509720 1500 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20261115 21:02:32.514719 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 1500
I20261115 21:02:32.514822 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20261115 21:02:32.516573 1628 heartbeater.cc:345] Connected to a master server at 127.29.135.254:33623
I20261115 21:02:32.516798 1628 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:32.516987 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.194:0
--local_ip_for_outbound_sockets=127.29.135.194
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:33623
--builtin_ntp_servers=127.29.135.212:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261115 21:02:32.517316 1628 heartbeater.cc:508] Master 127.29.135.254:33623 requested a full tablet report, sending...
I20261115 21:02:32.519048 1439 ts_manager.cc:210] Registered new tserver with Master: f21762a3da024f72b68d0a6abebfcc2d (127.29.135.193:40275)
I20261115 21:02:32.520736 1439 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.193:59055
W20261115 21:02:32.626161 1631 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:32.626387 1631 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:32.626410 1631 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:32.626451 1631 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:32.630260 1631 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:32.630337 1631 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:32.630374 1631 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:32.630442 1631 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:32.630471 1631 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.194
I20261115 21:02:32.636895 1631 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.29.135.194
--webserver_port=0
--tserver_master_addrs=127.29.135.254:33623
--never_fsync=true
--heap_profile_path=/tmp/kudu.1631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:32.637748 1631 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:32.638468 1631 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:32.645665 1636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:32.645794 1637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:32.645931 1631 server_base.cc:1061] running on GCE node
W20261115 21:02:32.645795 1639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:32.646426 1631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:32.647040 1631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:32.648262 1631 hybrid_clock.cc:648] HybridClock initialized: now 1794776552648250 us; error 31 us; skew 500 ppm
I20261115 21:02:32.650188 1631 webserver.cc:577] Webserver started at http://127.29.135.194:44979/ using document root <none> and password file <none>
I20261115 21:02:32.650835 1631 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:32.650887 1631 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:32.651100 1631 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:32.652503 1631 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "ca43e902b1aa4b0ca27a30ad61720e1f"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.652938 1631 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "ca43e902b1aa4b0ca27a30ad61720e1f"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.658209 1631 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20261115 21:02:32.662058 1645 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.663830 1631 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20261115 21:02:32.664050 1631 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "ca43e902b1aa4b0ca27a30ad61720e1f"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.664167 1631 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:32.678433 1631 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:32.678992 1631 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:32.679134 1631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:32.679661 1631 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:32.680718 1631 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:32.680773 1631 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.680826 1631 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:32.680843 1631 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.690306 1631 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.194:39037
I20261115 21:02:32.690322 1758 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.194:39037 every 8 connection(s)
I20261115 21:02:32.691509 1631 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261115 21:02:32.693743 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 1631
I20261115 21:02:32.693874 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20261115 21:02:32.697184 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.195:0
--local_ip_for_outbound_sockets=127.29.135.195
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:33623
--builtin_ntp_servers=127.29.135.212:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261115 21:02:32.702796 1759 heartbeater.cc:345] Connected to a master server at 127.29.135.254:33623
I20261115 21:02:32.703058 1759 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:32.703617 1759 heartbeater.cc:508] Master 127.29.135.254:33623 requested a full tablet report, sending...
I20261115 21:02:32.705000 1439 ts_manager.cc:210] Registered new tserver with Master: ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037)
I20261115 21:02:32.705976 1439 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.194:37527
W20261115 21:02:32.827621 1762 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:32.827821 1762 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:32.827849 1762 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:32.827893 1762 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:32.831311 1762 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:32.831368 1762 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:32.831403 1762 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:32.831475 1762 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:32.831504 1762 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.195
I20261115 21:02:32.835541 1762 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.29.135.195
--webserver_port=0
--tserver_master_addrs=127.29.135.254:33623
--never_fsync=true
--heap_profile_path=/tmp/kudu.1762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:32.836278 1762 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:32.836942 1762 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:32.842635 1767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:32.842644 1768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:32.842679 1770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:32.843014 1762 server_base.cc:1061] running on GCE node
I20261115 21:02:32.843408 1762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:32.843993 1762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:32.845209 1762 hybrid_clock.cc:648] HybridClock initialized: now 1794776552845137 us; error 75 us; skew 500 ppm
I20261115 21:02:32.847373 1762 webserver.cc:577] Webserver started at http://127.29.135.195:40319/ using document root <none> and password file <none>
I20261115 21:02:32.848008 1762 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:32.848062 1762 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:32.848278 1762 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:32.849613 1762 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "47b6605ca21d42b3a0a81641d44a8bde"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.850059 1762 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "47b6605ca21d42b3a0a81641d44a8bde"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.854085 1762 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20261115 21:02:32.856963 1776 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.858086 1762 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:32.858199 1762 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "47b6605ca21d42b3a0a81641d44a8bde"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:32.858302 1762 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:32.873417 1762 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:32.873885 1762 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:32.874018 1762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:32.874576 1762 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:32.875689 1762 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:32.875749 1762 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.875814 1762 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:32.875845 1762 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:32.884233 1762 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.195:38971
I20261115 21:02:32.884290 1890 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.195:38971 every 8 connection(s)
I20261115 21:02:32.885231 1762 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261115 21:02:32.892531 1891 heartbeater.cc:345] Connected to a master server at 127.29.135.254:33623
I20261115 21:02:32.892762 1891 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:32.893231 1891 heartbeater.cc:508] Master 127.29.135.254:33623 requested a full tablet report, sending...
I20261115 21:02:32.894294 1439 ts_manager.cc:210] Registered new tserver with Master: 47b6605ca21d42b3a0a81641d44a8bde (127.29.135.195:38971)
I20261115 21:02:32.894321 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 1762
I20261115 21:02:32.894423 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20261115 21:02:32.894883 1439 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.195:40913
I20261115 21:02:32.896450 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.196:0
--local_ip_for_outbound_sockets=127.29.135.196
--webserver_interface=127.29.135.196
--webserver_port=0
--tserver_master_addrs=127.29.135.254:33623
--builtin_ntp_servers=127.29.135.212:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261115 21:02:32.999048 1894 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:32.999270 1894 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:32.999298 1894 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:32.999365 1894 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:33.002936 1894 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:33.003003 1894 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:33.003041 1894 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:33.003136 1894 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:33.003170 1894 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.196
I20261115 21:02:33.007479 1894 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.29.135.196
--webserver_port=0
--tserver_master_addrs=127.29.135.254:33623
--never_fsync=true
--heap_profile_path=/tmp/kudu.1894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.196
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:33.008317 1894 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:33.009032 1894 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:33.014801 1902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:33.014802 1900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:33.014802 1899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:33.014986 1894 server_base.cc:1061] running on GCE node
I20261115 21:02:33.015460 1894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:33.015996 1894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:33.017185 1894 hybrid_clock.cc:648] HybridClock initialized: now 1794776553017170 us; error 29 us; skew 500 ppm
I20261115 21:02:33.018976 1894 webserver.cc:577] Webserver started at http://127.29.135.196:34651/ using document root <none> and password file <none>
I20261115 21:02:33.019546 1894 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:33.019591 1894 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:33.019789 1894 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261115 21:02:33.021044 1894 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "ec1c0b49f11747fca367978201a80e4f"
format_stamp: "Formatted at 2026-11-15 21:02:33 on dist-test-slave-xh31"
I20261115 21:02:33.021481 1894 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "ec1c0b49f11747fca367978201a80e4f"
format_stamp: "Formatted at 2026-11-15 21:02:33 on dist-test-slave-xh31"
I20261115 21:02:33.024724 1894 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20261115 21:02:33.026862 1908 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:33.027784 1894 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20261115 21:02:33.027889 1894 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "ec1c0b49f11747fca367978201a80e4f"
format_stamp: "Formatted at 2026-11-15 21:02:33 on dist-test-slave-xh31"
I20261115 21:02:33.027968 1894 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:33.039697 1894 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:33.040139 1894 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:33.040278 1894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:33.040750 1894 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:33.041707 1894 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261115 21:02:33.041752 1894 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:33.041808 1894 ts_tablet_manager.cc:662] Registered 0 tablets
I20261115 21:02:33.041838 1894 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:33.049862 1894 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.196:42611
I20261115 21:02:33.049913 2021 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.196:42611 every 8 connection(s)
I20261115 21:02:33.050891 1894 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261115 21:02:33.051867 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 1894
I20261115 21:02:33.051978 30239 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20261115 21:02:33.058161 2022 heartbeater.cc:345] Connected to a master server at 127.29.135.254:33623
I20261115 21:02:33.058411 2022 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:33.058907 2022 heartbeater.cc:508] Master 127.29.135.254:33623 requested a full tablet report, sending...
I20261115 21:02:33.060187 1439 ts_manager.cc:210] Registered new tserver with Master: ec1c0b49f11747fca367978201a80e4f (127.29.135.196:42611)
I20261115 21:02:33.060916 1439 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.196:43577
I20261115 21:02:33.065857 30239 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261115 21:02:33.077064 1439 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40294:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20261115 21:02:33.096575 1693 tablet_service.cc:1500] Processing CreateTablet for tablet 5c78fbada8af4df1b91415da7df9db88 (DEFAULT_TABLE table=TestTable [id=2c795382e81a43bfad9e1b2210c456eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:33.096576 1956 tablet_service.cc:1500] Processing CreateTablet for tablet 5c78fbada8af4df1b91415da7df9db88 (DEFAULT_TABLE table=TestTable [id=2c795382e81a43bfad9e1b2210c456eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:33.096576 1562 tablet_service.cc:1500] Processing CreateTablet for tablet 5c78fbada8af4df1b91415da7df9db88 (DEFAULT_TABLE table=TestTable [id=2c795382e81a43bfad9e1b2210c456eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20261115 21:02:33.097473 1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c78fbada8af4df1b91415da7df9db88. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:33.097498 1956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c78fbada8af4df1b91415da7df9db88. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:33.097498 1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c78fbada8af4df1b91415da7df9db88. 1 dirs total, 0 dirs full, 0 dirs failed
I20261115 21:02:33.102579 2039 tablet_bootstrap.cc:492] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Bootstrap starting.
I20261115 21:02:33.103670 2040 tablet_bootstrap.cc:492] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Bootstrap starting.
I20261115 21:02:33.103663 2041 tablet_bootstrap.cc:492] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Bootstrap starting.
I20261115 21:02:33.104939 2039 tablet_bootstrap.cc:654] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:33.105666 2039 log.cc:826] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:33.105880 2040 tablet_bootstrap.cc:654] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:33.105880 2041 tablet_bootstrap.cc:654] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Neither blocks nor log segments found. Creating new log.
I20261115 21:02:33.106751 2040 log.cc:826] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:33.106751 2041 log.cc:826] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:33.106983 2039 tablet_bootstrap.cc:492] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: No bootstrap required, opened a new log
I20261115 21:02:33.107168 2039 ts_tablet_manager.cc:1449] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20261115 21:02:33.107949 2041 tablet_bootstrap.cc:492] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: No bootstrap required, opened a new log
I20261115 21:02:33.108085 2040 tablet_bootstrap.cc:492] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: No bootstrap required, opened a new log
I20261115 21:02:33.108136 2041 ts_tablet_manager.cc:1449] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20261115 21:02:33.108276 2040 ts_tablet_manager.cc:1449] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20261115 21:02:33.110280 2041 raft_consensus.cc:360] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:33.110446 2041 raft_consensus.cc:386] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:33.110477 2041 raft_consensus.cc:748] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec1c0b49f11747fca367978201a80e4f, State: Initialized, Role: FOLLOWER
I20261115 21:02:33.110885 2041 consensus_queue.cc:261] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:33.111168 2039 raft_consensus.cc:360] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:33.111168 2040 raft_consensus.cc:360] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:33.111356 2040 raft_consensus.cc:386] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:33.111393 2039 raft_consensus.cc:386] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261115 21:02:33.111435 2040 raft_consensus.cc:748] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f21762a3da024f72b68d0a6abebfcc2d, State: Initialized, Role: FOLLOWER
I20261115 21:02:33.111465 2039 raft_consensus.cc:748] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca43e902b1aa4b0ca27a30ad61720e1f, State: Initialized, Role: FOLLOWER
I20261115 21:02:33.111647 2041 ts_tablet_manager.cc:1480] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20261115 21:02:33.111680 2022 heartbeater.cc:500] Master 127.29.135.254:33623 was elected leader, sending a full tablet report...
I20261115 21:02:33.111842 2039 consensus_queue.cc:261] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:33.111840 2040 consensus_queue.cc:261] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:33.112470 1628 heartbeater.cc:500] Master 127.29.135.254:33623 was elected leader, sending a full tablet report...
I20261115 21:02:33.112584 2039 ts_tablet_manager.cc:1480] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20261115 21:02:33.112709 1759 heartbeater.cc:500] Master 127.29.135.254:33623 was elected leader, sending a full tablet report...
I20261115 21:02:33.113653 2040 ts_tablet_manager.cc:1480] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
W20261115 21:02:33.151662 2023 tablet.cc:2441] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261115 21:02:33.194587 1760 tablet.cc:2441] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261115 21:02:33.211758 1629 tablet.cc:2441] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261115 21:02:33.231253 2046 raft_consensus.cc:499] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261115 21:02:33.231451 2046 raft_consensus.cc:521] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:33.232621 2046 leader_election.cc:290] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f21762a3da024f72b68d0a6abebfcc2d (127.29.135.193:40275), ec1c0b49f11747fca367978201a80e4f (127.29.135.196:42611)
I20261115 21:02:33.236198 1582 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f21762a3da024f72b68d0a6abebfcc2d" is_pre_election: true
I20261115 21:02:33.236498 1582 raft_consensus.cc:2490] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca43e902b1aa4b0ca27a30ad61720e1f in term 0.
I20261115 21:02:33.236739 1976 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec1c0b49f11747fca367978201a80e4f" is_pre_election: true
I20261115 21:02:33.236967 1646 leader_election.cc:304] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca43e902b1aa4b0ca27a30ad61720e1f, f21762a3da024f72b68d0a6abebfcc2d; no voters:
I20261115 21:02:33.236990 1976 raft_consensus.cc:2490] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca43e902b1aa4b0ca27a30ad61720e1f in term 0.
I20261115 21:02:33.237339 2046 raft_consensus.cc:2826] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261115 21:02:33.237408 2046 raft_consensus.cc:499] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261115 21:02:33.237458 2046 raft_consensus.cc:3096] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:33.238279 2046 raft_consensus.cc:521] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:33.238610 2046 leader_election.cc:290] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [CANDIDATE]: Term 1 election: Requested vote from peers f21762a3da024f72b68d0a6abebfcc2d (127.29.135.193:40275), ec1c0b49f11747fca367978201a80e4f (127.29.135.196:42611)
I20261115 21:02:33.238910 1582 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f21762a3da024f72b68d0a6abebfcc2d"
I20261115 21:02:33.238927 1976 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec1c0b49f11747fca367978201a80e4f"
I20261115 21:02:33.239010 1582 raft_consensus.cc:3096] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:33.239041 1976 raft_consensus.cc:3096] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 0 FOLLOWER]: Advancing to term 1
I20261115 21:02:33.239867 1582 raft_consensus.cc:2490] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca43e902b1aa4b0ca27a30ad61720e1f in term 1.
I20261115 21:02:33.239890 1976 raft_consensus.cc:2490] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca43e902b1aa4b0ca27a30ad61720e1f in term 1.
I20261115 21:02:33.240105 1646 leader_election.cc:304] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca43e902b1aa4b0ca27a30ad61720e1f, f21762a3da024f72b68d0a6abebfcc2d; no voters:
I20261115 21:02:33.240314 2046 raft_consensus.cc:2826] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 1 FOLLOWER]: Leader election won for term 1
I20261115 21:02:33.240515 2046 raft_consensus.cc:705] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 1 LEADER]: Becoming Leader. State: Replica: ca43e902b1aa4b0ca27a30ad61720e1f, State: Running, Role: LEADER
I20261115 21:02:33.240823 2046 consensus_queue.cc:238] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:33.244242 2046 consensus_queue.cc:1049] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:33.251842 2046 consensus_queue.cc:1049] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261115 21:02:33.244405 1440 catalog_manager.cc:5840] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f reported cstate change: term changed from 0 to 1, leader changed from <none> to ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194). New cstate: current_term: 1 leader_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } health_report { overall_health: UNKNOWN } } }
I20261115 21:02:33.277318 30239 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261115 21:02:33.278877 30239 ts_itest-base.cc:246] Waiting for 1 tablets on tserver f21762a3da024f72b68d0a6abebfcc2d to finish bootstrapping
I20261115 21:02:33.287695 30239 ts_itest-base.cc:246] Waiting for 1 tablets on tserver ca43e902b1aa4b0ca27a30ad61720e1f to finish bootstrapping
I20261115 21:02:33.292673 30239 ts_itest-base.cc:246] Waiting for 1 tablets on tserver ec1c0b49f11747fca367978201a80e4f to finish bootstrapping
I20261115 21:02:33.297159 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 1631
I20261115 21:02:33.303123 30239 external_mini_cluster.cc:1372] Running /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
/tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.135.194:39037
--local_ip_for_outbound_sockets=127.29.135.194
--tserver_master_addrs=127.29.135.254:33623
--webserver_port=44979
--webserver_interface=127.29.135.194
--builtin_ntp_servers=127.29.135.212:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000
--tablet_bootstrap_inject_latency_ms=3000 with env {}
W20261115 21:02:33.408221 2059 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261115 21:02:33.408418 2059 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261115 21:02:33.408457 2059 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261115 21:02:33.408519 2059 flags.cc:432] Enabled unsafe flag: --tablet_bootstrap_inject_latency_ms=3000
W20261115 21:02:33.408550 2059 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261115 21:02:33.411912 2059 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261115 21:02:33.411959 2059 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261115 21:02:33.412002 2059 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261115 21:02:33.412096 2059 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261115 21:02:33.412127 2059 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.135.194
I20261115 21:02:33.416079 2059 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.135.212:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.135.194:39037
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.29.135.194
--webserver_port=44979
--tserver_master_addrs=127.29.135.254:33623
--tablet_bootstrap_inject_latency_ms=3000
--never_fsync=true
--heap_profile_path=/tmp/kudu.2059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.135.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type DEBUG
built by None at 15 Nov 2026 20:43:16 UTC on bdcb31816ec0
build id 14799
I20261115 21:02:33.416803 2059 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261115 21:02:33.417474 2059 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261115 21:02:33.422923 2067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261115 21:02:33.422900 2064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:33.423026 2059 server_base.cc:1061] running on GCE node
W20261115 21:02:33.422921 2065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261115 21:02:33.423478 2059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261115 21:02:33.424012 2059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261115 21:02:33.425182 2059 hybrid_clock.cc:648] HybridClock initialized: now 1794776553425157 us; error 36 us; skew 500 ppm
I20261115 21:02:33.426738 2059 webserver.cc:577] Webserver started at http://127.29.135.194:44979/ using document root <none> and password file <none>
I20261115 21:02:33.427352 2059 fs_manager.cc:362] Metadata directory not provided
I20261115 21:02:33.427431 2059 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261115 21:02:33.430588 2059 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20261115 21:02:33.432576 2073 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261115 21:02:33.433481 2059 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20261115 21:02:33.433578 2059 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "ca43e902b1aa4b0ca27a30ad61720e1f"
format_stamp: "Formatted at 2026-11-15 21:02:32 on dist-test-slave-xh31"
I20261115 21:02:33.433935 2059 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261115 21:02:33.446564 2059 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261115 21:02:33.446961 2059 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261115 21:02:33.447070 2059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261115 21:02:33.447471 2059 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261115 21:02:33.448532 2080 ts_tablet_manager.cc:588] Loading tablet metadata (0/1 complete)
I20261115 21:02:33.452054 2059 ts_tablet_manager.cc:631] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20261115 21:02:33.452112 2059 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20261115 21:02:33.452163 2059 ts_tablet_manager.cc:646] Registering tablets (0/1 complete)
I20261115 21:02:33.453617 2059 ts_tablet_manager.cc:662] Registered 1 tablets
I20261115 21:02:33.453657 2059 ts_tablet_manager.cc:641] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
W20261115 21:02:33.453667 2080 ts_tablet_manager.cc:1428] Injecting 3000ms delay in tablet bootstrapping
I20261115 21:02:33.461637 2059 rpc_server.cc:307] RPC server started. Bound to: 127.29.135.194:39037
I20261115 21:02:33.461669 2187 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.135.194:39037 every 8 connection(s)
I20261115 21:02:33.462689 2059 server_base.cc:1193] Dumped server information to /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261115 21:02:33.468828 30239 external_mini_cluster.cc:1434] Started /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu as pid 2059
I20261115 21:02:33.469753 2188 heartbeater.cc:345] Connected to a master server at 127.29.135.254:33623
I20261115 21:02:33.469970 2188 heartbeater.cc:462] Registering TS with master...
I20261115 21:02:33.470479 2188 heartbeater.cc:508] Master 127.29.135.254:33623 requested a full tablet report, sending...
I20261115 21:02:33.472087 1440 ts_manager.cc:210] Re-registered known tserver with Master: ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037)
I20261115 21:02:33.472965 1440 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.29.135.194:48567
I20261115 21:02:33.896806 1891 heartbeater.cc:500] Master 127.29.135.254:33623 was elected leader, sending a full tablet report...
I20261115 21:02:34.475006 2188 heartbeater.cc:500] Master 127.29.135.254:33623 was elected leader, sending a full tablet report...
I20261115 21:02:34.757436 2193 raft_consensus.cc:499] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ca43e902b1aa4b0ca27a30ad61720e1f)
I20261115 21:02:34.757573 2193 raft_consensus.cc:521] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:34.758236 2194 raft_consensus.cc:499] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ca43e902b1aa4b0ca27a30ad61720e1f)
I20261115 21:02:34.758355 2194 raft_consensus.cc:521] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:34.758630 2193 leader_election.cc:290] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037), ec1c0b49f11747fca367978201a80e4f (127.29.135.196:42611)
I20261115 21:02:34.759459 2194 leader_election.cc:290] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f21762a3da024f72b68d0a6abebfcc2d (127.29.135.193:40275), ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037)
I20261115 21:02:34.763226 1976 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "f21762a3da024f72b68d0a6abebfcc2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec1c0b49f11747fca367978201a80e4f" is_pre_election: true
I20261115 21:02:34.763350 1976 raft_consensus.cc:2490] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f21762a3da024f72b68d0a6abebfcc2d in term 1.
I20261115 21:02:34.763777 1518 leader_election.cc:304] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec1c0b49f11747fca367978201a80e4f, f21762a3da024f72b68d0a6abebfcc2d; no voters:
I20261115 21:02:34.763873 1582 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "ec1c0b49f11747fca367978201a80e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f21762a3da024f72b68d0a6abebfcc2d" is_pre_election: true
I20261115 21:02:34.763988 1582 raft_consensus.cc:2490] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec1c0b49f11747fca367978201a80e4f in term 1.
I20261115 21:02:34.764060 2193 raft_consensus.cc:2826] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261115 21:02:34.764114 2193 raft_consensus.cc:499] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 1 FOLLOWER]: Starting leader election (detected failure of leader ca43e902b1aa4b0ca27a30ad61720e1f)
I20261115 21:02:34.764153 2193 raft_consensus.cc:3096] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 1 FOLLOWER]: Advancing to term 2
I20261115 21:02:34.764377 1909 leader_election.cc:304] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec1c0b49f11747fca367978201a80e4f, f21762a3da024f72b68d0a6abebfcc2d; no voters:
I20261115 21:02:34.764598 2194 raft_consensus.cc:2826] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261115 21:02:34.764665 2194 raft_consensus.cc:499] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ca43e902b1aa4b0ca27a30ad61720e1f)
I20261115 21:02:34.764703 2194 raft_consensus.cc:3096] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 1 FOLLOWER]: Advancing to term 2
I20261115 21:02:34.763957 2142 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "f21762a3da024f72b68d0a6abebfcc2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" is_pre_election: true
I20261115 21:02:34.764125 2141 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "ec1c0b49f11747fca367978201a80e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" is_pre_election: true
I20261115 21:02:34.765125 2193 raft_consensus.cc:521] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:34.765431 2193 leader_election.cc:290] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 2 election: Requested vote from peers ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037), ec1c0b49f11747fca367978201a80e4f (127.29.135.196:42611)
I20261115 21:02:34.765686 2194 raft_consensus.cc:521] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:34.765802 1976 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "f21762a3da024f72b68d0a6abebfcc2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec1c0b49f11747fca367978201a80e4f"
I20261115 21:02:34.765950 1976 raft_consensus.cc:2415] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f21762a3da024f72b68d0a6abebfcc2d in current term 2: Already voted for candidate ec1c0b49f11747fca367978201a80e4f in this term.
I20261115 21:02:34.765964 2141 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "f21762a3da024f72b68d0a6abebfcc2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f"
I20261115 21:02:34.766036 2194 leader_election.cc:290] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [CANDIDATE]: Term 2 election: Requested vote from peers f21762a3da024f72b68d0a6abebfcc2d (127.29.135.193:40275), ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037)
W20261115 21:02:34.766120 1516 leader_election.cc:343] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037): Illegal state: must be running to vote when last-logged opid is not known
W20261115 21:02:34.766426 1516 leader_election.cc:343] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037): Illegal state: must be running to vote when last-logged opid is not known
I20261115 21:02:34.766448 2141 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "ec1c0b49f11747fca367978201a80e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f"
I20261115 21:02:34.766477 1582 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "ec1c0b49f11747fca367978201a80e4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f21762a3da024f72b68d0a6abebfcc2d"
I20261115 21:02:34.766515 1516 leader_election.cc:304] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f21762a3da024f72b68d0a6abebfcc2d; no voters: ca43e902b1aa4b0ca27a30ad61720e1f, ec1c0b49f11747fca367978201a80e4f
I20261115 21:02:34.766631 1582 raft_consensus.cc:2415] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ec1c0b49f11747fca367978201a80e4f in current term 2: Already voted for candidate f21762a3da024f72b68d0a6abebfcc2d in this term.
I20261115 21:02:34.766705 2193 raft_consensus.cc:2771] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20261115 21:02:34.766846 1910 leader_election.cc:343] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037): Illegal state: must be running to vote when last-logged opid is not known
I20261115 21:02:34.766963 1909 leader_election.cc:304] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec1c0b49f11747fca367978201a80e4f; no voters: ca43e902b1aa4b0ca27a30ad61720e1f, f21762a3da024f72b68d0a6abebfcc2d
I20261115 21:02:34.767103 2194 raft_consensus.cc:2771] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20261115 21:02:36.377768 2204 raft_consensus.cc:499] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261115 21:02:36.377900 2204 raft_consensus.cc:521] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:36.378283 2204 leader_election.cc:290] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037), ec1c0b49f11747fca367978201a80e4f (127.29.135.196:42611)
I20261115 21:02:36.378765 2141 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "f21762a3da024f72b68d0a6abebfcc2d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" is_pre_election: true
I20261115 21:02:36.378773 1976 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "f21762a3da024f72b68d0a6abebfcc2d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec1c0b49f11747fca367978201a80e4f" is_pre_election: true
I20261115 21:02:36.378913 1976 raft_consensus.cc:2490] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f21762a3da024f72b68d0a6abebfcc2d in term 2.
W20261115 21:02:36.379181 1516 leader_election.cc:343] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037): Illegal state: must be running to vote when last-logged opid is not known
I20261115 21:02:36.379308 1518 leader_election.cc:304] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec1c0b49f11747fca367978201a80e4f, f21762a3da024f72b68d0a6abebfcc2d; no voters: ca43e902b1aa4b0ca27a30ad61720e1f
I20261115 21:02:36.379478 2204 raft_consensus.cc:2826] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20261115 21:02:36.379534 2204 raft_consensus.cc:499] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261115 21:02:36.379560 2204 raft_consensus.cc:3096] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 2 FOLLOWER]: Advancing to term 3
I20261115 21:02:36.380330 2204 raft_consensus.cc:521] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:36.380620 2204 leader_election.cc:290] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 3 election: Requested vote from peers ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037), ec1c0b49f11747fca367978201a80e4f (127.29.135.196:42611)
I20261115 21:02:36.380913 2141 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "f21762a3da024f72b68d0a6abebfcc2d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f"
I20261115 21:02:36.380923 1976 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5c78fbada8af4df1b91415da7df9db88" candidate_uuid: "f21762a3da024f72b68d0a6abebfcc2d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec1c0b49f11747fca367978201a80e4f"
I20261115 21:02:36.381037 1976 raft_consensus.cc:3096] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 2 FOLLOWER]: Advancing to term 3
W20261115 21:02:36.381207 1516 leader_election.cc:343] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037): Illegal state: must be running to vote when last-logged opid is not known
I20261115 21:02:36.381814 1976 raft_consensus.cc:2490] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f21762a3da024f72b68d0a6abebfcc2d in term 3.
I20261115 21:02:36.382042 1518 leader_election.cc:304] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec1c0b49f11747fca367978201a80e4f, f21762a3da024f72b68d0a6abebfcc2d; no voters: ca43e902b1aa4b0ca27a30ad61720e1f
I20261115 21:02:36.382184 2204 raft_consensus.cc:2826] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 3 FOLLOWER]: Leader election won for term 3
I20261115 21:02:36.382359 2204 raft_consensus.cc:705] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [term 3 LEADER]: Becoming Leader. State: Replica: f21762a3da024f72b68d0a6abebfcc2d, State: Running, Role: LEADER
I20261115 21:02:36.382726 2204 consensus_queue.cc:238] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:36.384577 1976 raft_consensus.cc:1285] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f [term 3 FOLLOWER]: Refusing update from remote peer f21762a3da024f72b68d0a6abebfcc2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20261115 21:02:36.384995 2204 consensus_queue.cc:1049] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20261115 21:02:36.385360 1516 consensus_peers.cc:597] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d -> Peer ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194:39037): Couldn't send request to peer ca43e902b1aa4b0ca27a30ad61720e1f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20261115 21:02:36.385249 1440 catalog_manager.cc:5840] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d reported cstate change: term changed from 1 to 3, leader changed from ca43e902b1aa4b0ca27a30ad61720e1f (127.29.135.194) to f21762a3da024f72b68d0a6abebfcc2d (127.29.135.193). New cstate: current_term: 3 leader_uuid: "f21762a3da024f72b68d0a6abebfcc2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } health_report { overall_health: UNKNOWN } } }
I20261115 21:02:36.454138 2080 tablet_bootstrap.cc:492] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Bootstrap starting.
I20261115 21:02:36.460727 2080 log.cc:826] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Log is configured to *not* fsync() on all Append() calls
I20261115 21:02:36.462874 2080 tablet_bootstrap.cc:492] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261115 21:02:36.463151 2080 tablet_bootstrap.cc:492] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Bootstrap complete.
I20261115 21:02:36.463495 2080 ts_tablet_manager.cc:1449] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20261115 21:02:36.465116 2080 raft_consensus.cc:360] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:36.465384 2080 raft_consensus.cc:748] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca43e902b1aa4b0ca27a30ad61720e1f, State: Initialized, Role: FOLLOWER
I20261115 21:02:36.465755 2080 consensus_queue.cc:261] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f21762a3da024f72b68d0a6abebfcc2d" member_type: VOTER last_known_addr { host: "127.29.135.193" port: 40275 } } peers { permanent_uuid: "ca43e902b1aa4b0ca27a30ad61720e1f" member_type: VOTER last_known_addr { host: "127.29.135.194" port: 39037 } } peers { permanent_uuid: "ec1c0b49f11747fca367978201a80e4f" member_type: VOTER last_known_addr { host: "127.29.135.196" port: 42611 } }
I20261115 21:02:36.466377 2080 ts_tablet_manager.cc:1480] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20261115 21:02:36.797431 2141 raft_consensus.cc:3096] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f [term 1 FOLLOWER]: Advancing to term 3
I20261115 21:02:37.126673 30239 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20261115 21:02:37.126938 30239 raft_config_change-itest.cc:610] Waiting until we've written at least 4MB...
I20261115 21:02:37.171365 2189 maintenance_manager.cc:419] P ca43e902b1aa4b0ca27a30ad61720e1f: Scheduling FlushMRSOp(5c78fbada8af4df1b91415da7df9db88): 1577944 bytes log retention, and flush 1958153 bytes memory
I20261115 21:02:37.202812 2078 maintenance_manager.cc:643] P ca43e902b1aa4b0ca27a30ad61720e1f: FlushMRSOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.031s user 0.027s sys 0.004s Metrics: {"bytes_written":135803,"cfile_init":1,"compiler_manager_pool.queue_time_us":1907,"drs_written":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4,"lbm_write_time_us":603,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":12,"thread_start_us":225,"threads_started":2}
I20261115 21:02:37.203719 2189 maintenance_manager.cc:419] P ca43e902b1aa4b0ca27a30ad61720e1f: Scheduling LogGCOp(5c78fbada8af4df1b91415da7df9db88): free 1577944 bytes of WAL
I20261115 21:02:37.203958 2078 log_reader.cc:385] T 5c78fbada8af4df1b91415da7df9db88: removed 2 log segments from log reader
I20261115 21:02:37.204011 2078 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000001 (ops 1-8)
I20261115 21:02:37.204077 2078 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000002 (ops 9-14)
I20261115 21:02:37.204579 2078 maintenance_manager.cc:643] P ca43e902b1aa4b0ca27a30ad61720e1f: LogGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261115 21:02:37.204872 2189 maintenance_manager.cc:419] P ca43e902b1aa4b0ca27a30ad61720e1f: Scheduling UndoDeltaBlockGCOp(5c78fbada8af4df1b91415da7df9db88): 274 bytes on disk
I20261115 21:02:37.205408 2078 maintenance_manager.cc:643] P ca43e902b1aa4b0ca27a30ad61720e1f: UndoDeltaBlockGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":52,"lbm_reads_lt_1ms":4}
I20261115 21:02:37.225915 1629 maintenance_manager.cc:419] P f21762a3da024f72b68d0a6abebfcc2d: Scheduling FlushMRSOp(5c78fbada8af4df1b91415da7df9db88): 4602274 bytes log retention, and flush 6119689 bytes memory
I20261115 21:02:37.236104 30239 raft_config_change-itest.cc:616] Waiting for log GC on f21762a3da024f72b68d0a6abebfcc2d
I20261115 21:02:37.265362 2023 maintenance_manager.cc:419] P ec1c0b49f11747fca367978201a80e4f: Scheduling FlushMRSOp(5c78fbada8af4df1b91415da7df9db88): 4339184 bytes log retention, and flush 6119689 bytes memory
I20261115 21:02:37.282589 1519 maintenance_manager.cc:643] P f21762a3da024f72b68d0a6abebfcc2d: FlushMRSOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.056s user 0.052s sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":210,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":622,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":36,"thread_start_us":212,"threads_started":2}
I20261115 21:02:37.283497 1629 maintenance_manager.cc:419] P f21762a3da024f72b68d0a6abebfcc2d: Scheduling LogGCOp(5c78fbada8af4df1b91415da7df9db88): free 4602274 bytes of WAL
I20261115 21:02:37.283751 1519 log_reader.cc:385] T 5c78fbada8af4df1b91415da7df9db88: removed 5 log segments from log reader
I20261115 21:02:37.283828 1519 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000001 (ops 1-9)
I20261115 21:02:37.283944 1519 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000002 (ops 10-16)
I20261115 21:02:37.284009 1519 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000003 (ops 17-23)
I20261115 21:02:37.284070 1519 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000004 (ops 24-30)
I20261115 21:02:37.284130 1519 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P f21762a3da024f72b68d0a6abebfcc2d: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-0/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000005 (ops 31-37)
I20261115 21:02:37.285501 1519 maintenance_manager.cc:643] P f21762a3da024f72b68d0a6abebfcc2d: LogGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {}
I20261115 21:02:37.285967 1629 maintenance_manager.cc:419] P f21762a3da024f72b68d0a6abebfcc2d: Scheduling UndoDeltaBlockGCOp(5c78fbada8af4df1b91415da7df9db88): 536 bytes on disk
I20261115 21:02:37.286525 1519 maintenance_manager.cc:643] P f21762a3da024f72b68d0a6abebfcc2d: UndoDeltaBlockGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":51,"lbm_reads_lt_1ms":4}
I20261115 21:02:37.287184 30239 raft_config_change-itest.cc:620] Log GC complete on f21762a3da024f72b68d0a6abebfcc2d
I20261115 21:02:37.306134 2189 maintenance_manager.cc:419] P ca43e902b1aa4b0ca27a30ad61720e1f: Scheduling FlushMRSOp(5c78fbada8af4df1b91415da7df9db88): 2629902 bytes log retention, and flush 4038921 bytes memory
I20261115 21:02:37.339936 1913 maintenance_manager.cc:643] P ec1c0b49f11747fca367978201a80e4f: FlushMRSOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.074s user 0.058s sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":20819,"drs_written":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4,"lbm_write_time_us":591,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":36,"thread_start_us":247,"threads_started":2}
I20261115 21:02:37.340864 2023 maintenance_manager.cc:419] P ec1c0b49f11747fca367978201a80e4f: Scheduling LogGCOp(5c78fbada8af4df1b91415da7df9db88): free 2629823 bytes of WAL
I20261115 21:02:37.341131 1913 log_reader.cc:385] T 5c78fbada8af4df1b91415da7df9db88: removed 3 log segments from log reader
I20261115 21:02:37.341220 1913 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000001 (ops 1-9)
I20261115 21:02:37.341346 1913 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000002 (ops 10-16)
I20261115 21:02:37.341418 1913 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000003 (ops 17-22)
I20261115 21:02:37.342294 1913 maintenance_manager.cc:643] P ec1c0b49f11747fca367978201a80e4f: LogGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261115 21:02:37.342629 2023 maintenance_manager.cc:419] P ec1c0b49f11747fca367978201a80e4f: Scheduling UndoDeltaBlockGCOp(5c78fbada8af4df1b91415da7df9db88): 536 bytes on disk
I20261115 21:02:37.343160 1913 maintenance_manager.cc:643] P ec1c0b49f11747fca367978201a80e4f: UndoDeltaBlockGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":54,"lbm_reads_lt_1ms":4}
I20261115 21:02:37.356384 2078 maintenance_manager.cc:643] P ca43e902b1aa4b0ca27a30ad61720e1f: FlushMRSOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.050s user 0.031s sys 0.001s Metrics: {"bytes_written":135869,"cfile_init":1,"dirs.queue_time_us":40,"dirs.run_cpu_time_us":129,"dirs.run_wall_time_us":933,"drs_written":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4,"lbm_write_time_us":594,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":24}
I20261115 21:02:37.357263 2189 maintenance_manager.cc:419] P ca43e902b1aa4b0ca27a30ad61720e1f: Scheduling LogGCOp(5c78fbada8af4df1b91415da7df9db88): free 1840997 bytes of WAL
I20261115 21:02:37.357496 2078 log_reader.cc:385] T 5c78fbada8af4df1b91415da7df9db88: removed 3 log segments from log reader
I20261115 21:02:37.357573 2078 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000003 (ops 15-19)
I20261115 21:02:37.357654 2078 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000004 (ops 20-24)
I20261115 21:02:37.357723 2078 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000005 (ops 25-28)
I20261115 21:02:37.358369 2078 maintenance_manager.cc:643] P ca43e902b1aa4b0ca27a30ad61720e1f: LogGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261115 21:02:37.358701 2189 maintenance_manager.cc:419] P ca43e902b1aa4b0ca27a30ad61720e1f: Scheduling UndoDeltaBlockGCOp(5c78fbada8af4df1b91415da7df9db88): 400 bytes on disk
I20261115 21:02:37.359068 2078 maintenance_manager.cc:643] P ca43e902b1aa4b0ca27a30ad61720e1f: UndoDeltaBlockGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4}
I20261115 21:02:37.359485 2189 maintenance_manager.cc:419] P ca43e902b1aa4b0ca27a30ad61720e1f: Scheduling CompactRowSetsOp(5c78fbada8af4df1b91415da7df9db88): perf score=0.946847
I20261115 21:02:37.410457 2078 maintenance_manager.cc:643] P ca43e902b1aa4b0ca27a30ad61720e1f: CompactRowSetsOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.051s user 0.050s sys 0.000s Metrics: {"bytes_written":136001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262392,"cfile_cache_miss":26,"cfile_cache_miss_bytes":264288,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":78,"dirs.run_cpu_time_us":147,"dirs.run_wall_time_us":1071,"drs_written":1,"lbm_read_time_us":701,"lbm_reads_lt_1ms":46,"lbm_write_time_us":606,"lbm_writes_lt_1ms":23,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":36,"thread_start_us":332,"threads_started":5}
I20261115 21:02:37.411114 2189 maintenance_manager.cc:419] P ca43e902b1aa4b0ca27a30ad61720e1f: Scheduling UndoDeltaBlockGCOp(5c78fbada8af4df1b91415da7df9db88): 536 bytes on disk
I20261115 21:02:37.411520 2078 maintenance_manager.cc:643] P ca43e902b1aa4b0ca27a30ad61720e1f: UndoDeltaBlockGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4}
I20261115 21:02:37.845155 2023 maintenance_manager.cc:419] P ec1c0b49f11747fca367978201a80e4f: Scheduling LogGCOp(5c78fbada8af4df1b91415da7df9db88): free 1709361 bytes of WAL
I20261115 21:02:37.845386 1913 log_reader.cc:385] T 5c78fbada8af4df1b91415da7df9db88: removed 2 log segments from log reader
I20261115 21:02:37.845438 1913 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000004 (ops 23-29)
I20261115 21:02:37.845479 1913 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ec1c0b49f11747fca367978201a80e4f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-3/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000005 (ops 30-35)
I20261115 21:02:37.845957 1913 maintenance_manager.cc:643] P ec1c0b49f11747fca367978201a80e4f: LogGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261115 21:02:37.913275 2189 maintenance_manager.cc:419] P ca43e902b1aa4b0ca27a30ad61720e1f: Scheduling LogGCOp(5c78fbada8af4df1b91415da7df9db88): free 788905 bytes of WAL
I20261115 21:02:37.913480 2078 log_reader.cc:385] T 5c78fbada8af4df1b91415da7df9db88: removed 1 log segments from log reader
I20261115 21:02:37.913522 2078 log.cc:1079] T 5c78fbada8af4df1b91415da7df9db88 P ca43e902b1aa4b0ca27a30ad61720e1f: Deleting log segment in path: /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0/raft_consensus-itest-cluster/ts-1/wal/wals/5c78fbada8af4df1b91415da7df9db88/wal-000000006 (ops 29-34)
I20261115 21:02:37.913787 2078 maintenance_manager.cc:643] P ca43e902b1aa4b0ca27a30ad61720e1f: LogGCOp(5c78fbada8af4df1b91415da7df9db88) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/integration-tests/raft_config_change-itest.cc:500: Failure
Value of: IsRaftConfigMember(added_replica_uuid, cstate.committed_config())
Actual: false
Expected: true
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20261115 21:03:07.432092 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 1500
I20261115 21:03:07.439054 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 2059
I20261115 21:03:07.445179 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 1762
I20261115 21:03:07.450441 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 1894
I20261115 21:03:07.456604 30239 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task2Yu5Ne/build/debug/bin/kudu with pid 1409
2026-11-15T21:03:07Z chronyd exiting
I20261115 21:03:07.473939 30239 test_util.cc:182] -----------------------------------------------
I20261115 21:03:07.474004 30239 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-task2Yu5Ne/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794776526014215-30239-0
[ FAILED ] SlowTabletBootstrapTest.TestFallBehindSlowBootstrap (35325 ms)
[----------] 3 tests from SlowTabletBootstrapTest (46825 ms total)
[----------] Global test environment tear-down
[==========] 6 tests from 2 test suites ran. (61450 ms total)
[ PASSED ] 5 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SlowTabletBootstrapTest.TestFallBehindSlowBootstrap
1 FAILED TEST