[==========] Running 6 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from RaftConfigChangeITest
[ RUN ] RaftConfigChangeITest.TestKudu2147
2026-11-17T09:06:06Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-17T09:06:06Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20261117 09:06:06.977499 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:39141
--webserver_interface=127.16.187.190
--webserver_port=0
--builtin_ntp_servers=127.16.187.148:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.187.190:39141 with env {}
W20261117 09:06:07.051847 17142 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:07.051999 17142 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:07.052016 17142 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:07.053517 17142 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261117 09:06:07.053560 17142 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:07.053571 17142 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261117 09:06:07.053581 17142 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261117 09:06:07.055136 17142 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.187.190:39141
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:39141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.187.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.17142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:07.055297 17142 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:07.055423 17142 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:07.057436 17147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:07.057440 17150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:07.057512 17142 server_base.cc:1061] running on GCE node
W20261117 09:06:07.057449 17148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:07.057819 17142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:07.058008 17142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:07.059157 17142 hybrid_clock.cc:648] HybridClock initialized: now 1794906367059145 us; error 30 us; skew 500 ppm
I20261117 09:06:07.060138 17142 webserver.cc:577] Webserver started at http://127.16.187.190:45271/ using document root <none> and password file <none>
I20261117 09:06:07.060314 17142 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:07.060352 17142 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:07.060441 17142 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:07.061252 17142 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/data/instance:
uuid: "12cf69e5386749c5b12f7e597efa93ef"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.061550 17142 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/wal/instance:
uuid: "12cf69e5386749c5b12f7e597efa93ef"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.062647 17142 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:07.063241 17156 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.063390 17142 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.063444 17142 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/wal
uuid: "12cf69e5386749c5b12f7e597efa93ef"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.063508 17142 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:07.076280 17142 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:07.076532 17142 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:07.076635 17142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:07.080585 17142 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.190:39141
I20261117 09:06:07.080610 17208 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.190:39141 every 8 connection(s)
I20261117 09:06:07.080930 17142 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/data/info.pb
I20261117 09:06:07.081182 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 17142
I20261117 09:06:07.081281 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/master-0/wal/instance
I20261117 09:06:07.081537 17209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:07.083835 17209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef: Bootstrap starting.
I20261117 09:06:07.084486 17209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:07.084697 17209 log.cc:826] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:07.085259 17209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef: No bootstrap required, opened a new log
I20261117 09:06:07.086390 17209 raft_consensus.cc:360] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12cf69e5386749c5b12f7e597efa93ef" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 39141 } }
I20261117 09:06:07.086551 17209 raft_consensus.cc:386] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:07.086580 17209 raft_consensus.cc:748] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12cf69e5386749c5b12f7e597efa93ef, State: Initialized, Role: FOLLOWER
I20261117 09:06:07.086683 17209 consensus_queue.cc:261] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12cf69e5386749c5b12f7e597efa93ef" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 39141 } }
I20261117 09:06:07.086756 17209 raft_consensus.cc:400] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261117 09:06:07.086817 17209 raft_consensus.cc:499] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261117 09:06:07.086863 17209 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:07.087488 17209 raft_consensus.cc:521] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12cf69e5386749c5b12f7e597efa93ef" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 39141 } }
I20261117 09:06:07.087597 17209 leader_election.cc:304] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12cf69e5386749c5b12f7e597efa93ef; no voters:
I20261117 09:06:07.087729 17209 leader_election.cc:290] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20261117 09:06:07.087769 17214 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:07.087891 17214 raft_consensus.cc:705] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [term 1 LEADER]: Becoming Leader. State: Replica: 12cf69e5386749c5b12f7e597efa93ef, State: Running, Role: LEADER
I20261117 09:06:07.087949 17209 sys_catalog.cc:565] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [sys.catalog]: configured and running, proceeding with master startup.
I20261117 09:06:07.088003 17214 consensus_queue.cc:238] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12cf69e5386749c5b12f7e597efa93ef" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 39141 } }
I20261117 09:06:07.088336 17215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12cf69e5386749c5b12f7e597efa93ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12cf69e5386749c5b12f7e597efa93ef" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 39141 } } }
I20261117 09:06:07.088356 17216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12cf69e5386749c5b12f7e597efa93ef. Latest consensus state: current_term: 1 leader_uuid: "12cf69e5386749c5b12f7e597efa93ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12cf69e5386749c5b12f7e597efa93ef" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 39141 } } }
I20261117 09:06:07.088496 17216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:07.088696 17215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:07.088914 17223 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261117 09:06:07.089311 17223 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261117 09:06:07.090672 17223 catalog_manager.cc:1398] Generated new cluster ID: be1073460d1f44499fbb392575bb3de2
I20261117 09:06:07.090723 17223 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261117 09:06:07.106997 17223 catalog_manager.cc:1421] Generated new certificate authority record
I20261117 09:06:07.107782 17223 catalog_manager.cc:1555] Loading token signing keys...
I20261117 09:06:07.121359 17223 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 12cf69e5386749c5b12f7e597efa93ef: Generated new TSK 0
I20261117 09:06:07.121562 17223 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261117 09:06:07.125471 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.129:0
--local_ip_for_outbound_sockets=127.16.187.129
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:39141
--builtin_ntp_servers=127.16.187.148:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000 with env {}
W20261117 09:06:07.200618 17233 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:07.200775 17233 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:07.200802 17233 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:07.202323 17233 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:07.202383 17233 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.129
I20261117 09:06:07.204003 17233 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:39141
--never_fsync=true
--heap_profile_path=/tmp/kudu.17233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:07.204209 17233 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:07.204376 17233 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:07.206709 17241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:07.206709 17239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:07.206962 17238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:07.207501 17233 server_base.cc:1061] running on GCE node
I20261117 09:06:07.207649 17233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:07.207832 17233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:07.208941 17233 hybrid_clock.cc:648] HybridClock initialized: now 1794906367208915 us; error 52 us; skew 500 ppm
I20261117 09:06:07.210078 17233 webserver.cc:577] Webserver started at http://127.16.187.129:37873/ using document root <none> and password file <none>
I20261117 09:06:07.210280 17233 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:07.210330 17233 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:07.210420 17233 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:07.211544 17233 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/data/instance:
uuid: "acbcb0f25f4848eaa9756e2eb85a97bd"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.211895 17233 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/wal/instance:
uuid: "acbcb0f25f4848eaa9756e2eb85a97bd"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.213260 17233 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:07.214078 17247 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.214268 17233 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:07.214349 17233 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/wal
uuid: "acbcb0f25f4848eaa9756e2eb85a97bd"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.214424 17233 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:07.225558 17233 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:07.225821 17233 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:07.225936 17233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:07.226130 17233 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:07.226449 17233 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:07.226482 17233 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.226516 17233 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:07.226560 17233 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.232304 17233 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.129:45937
I20261117 09:06:07.232401 17361 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.129:45937 every 8 connection(s)
I20261117 09:06:07.232676 17233 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/data/info.pb
I20261117 09:06:07.237685 17362 heartbeater.cc:345] Connected to a master server at 127.16.187.190:39141
I20261117 09:06:07.237794 17362 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:07.237988 17362 heartbeater.cc:508] Master 127.16.187.190:39141 requested a full tablet report, sending...
I20261117 09:06:07.238478 17173 ts_manager.cc:210] Registered new tserver with Master: acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129:45937)
I20261117 09:06:07.239217 17173 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.129:38583
I20261117 09:06:07.239624 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 17233
I20261117 09:06:07.239703 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/wal/instance
I20261117 09:06:07.240694 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.130:0
--local_ip_for_outbound_sockets=127.16.187.130
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:39141
--builtin_ntp_servers=127.16.187.148:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000 with env {}
W20261117 09:06:07.320112 17365 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:07.320250 17365 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:07.320266 17365 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:07.321777 17365 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:07.321825 17365 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.130
I20261117 09:06:07.323390 17365 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:39141
--never_fsync=true
--heap_profile_path=/tmp/kudu.17365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:07.323593 17365 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:07.323802 17365 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:07.325984 17370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:07.326072 17371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:07.326165 17365 server_base.cc:1061] running on GCE node
W20261117 09:06:07.326390 17373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:07.326587 17365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:07.326781 17365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:07.327907 17365 hybrid_clock.cc:648] HybridClock initialized: now 1794906367327892 us; error 27 us; skew 500 ppm
I20261117 09:06:07.328923 17365 webserver.cc:577] Webserver started at http://127.16.187.130:43997/ using document root <none> and password file <none>
I20261117 09:06:07.329094 17365 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:07.329133 17365 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:07.329216 17365 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:07.329985 17365 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/data/instance:
uuid: "d4291cda32014c93b90c840294450aa7"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.330278 17365 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/wal/instance:
uuid: "d4291cda32014c93b90c840294450aa7"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.331391 17365 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261117 09:06:07.332159 17380 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.332341 17365 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261117 09:06:07.332399 17365 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/wal
uuid: "d4291cda32014c93b90c840294450aa7"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.332451 17365 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:07.347470 17365 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:07.347723 17365 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:07.347831 17365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:07.348016 17365 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:07.348310 17365 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:07.348340 17365 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.348369 17365 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:07.348395 17365 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.353684 17365 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.130:38939
I20261117 09:06:07.353734 17493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.130:38939 every 8 connection(s)
I20261117 09:06:07.354013 17365 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/data/info.pb
I20261117 09:06:07.355515 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 17365
I20261117 09:06:07.355604 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-1/wal/instance
I20261117 09:06:07.356647 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.131:0
--local_ip_for_outbound_sockets=127.16.187.131
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:39141
--builtin_ntp_servers=127.16.187.148:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000 with env {}
I20261117 09:06:07.358134 17494 heartbeater.cc:345] Connected to a master server at 127.16.187.190:39141
I20261117 09:06:07.358215 17494 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:07.358397 17494 heartbeater.cc:508] Master 127.16.187.190:39141 requested a full tablet report, sending...
I20261117 09:06:07.358752 17173 ts_manager.cc:210] Registered new tserver with Master: d4291cda32014c93b90c840294450aa7 (127.16.187.130:38939)
I20261117 09:06:07.359123 17173 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.130:40557
W20261117 09:06:07.431280 17497 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:07.431412 17497 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:07.431430 17497 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:07.432905 17497 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:07.432957 17497 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.131
I20261117 09:06:07.434490 17497 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:39141
--never_fsync=true
--heap_profile_path=/tmp/kudu.17497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:07.434692 17497 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:07.434852 17497 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:07.437179 17505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:07.437265 17503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:07.437201 17502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:07.437451 17497 server_base.cc:1061] running on GCE node
I20261117 09:06:07.437574 17497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:07.437736 17497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:07.438913 17497 hybrid_clock.cc:648] HybridClock initialized: now 1794906367438865 us; error 31 us; skew 500 ppm
I20261117 09:06:07.439936 17497 webserver.cc:577] Webserver started at http://127.16.187.131:33889/ using document root <none> and password file <none>
I20261117 09:06:07.440119 17497 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:07.440162 17497 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:07.440254 17497 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:07.441067 17497 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/data/instance:
uuid: "5d8ecddcda974b718d9f5360545a674c"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.441386 17497 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/wal/instance:
uuid: "5d8ecddcda974b718d9f5360545a674c"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.442574 17497 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20261117 09:06:07.443333 17511 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.443509 17497 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:07.443570 17497 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/wal
uuid: "5d8ecddcda974b718d9f5360545a674c"
format_stamp: "Formatted at 2026-11-17 09:06:07 on dist-test-slave-x52s"
I20261117 09:06:07.443627 17497 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:07.468181 17497 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:07.468472 17497 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:07.468592 17497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:07.468796 17497 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:07.469130 17497 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:07.469166 17497 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.469201 17497 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:07.469228 17497 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:07.475245 17497 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.131:43889
I20261117 09:06:07.475293 17625 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.131:43889 every 8 connection(s)
I20261117 09:06:07.475605 17497 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/data/info.pb
I20261117 09:06:07.479780 17626 heartbeater.cc:345] Connected to a master server at 127.16.187.190:39141
I20261117 09:06:07.479866 17626 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:07.480041 17626 heartbeater.cc:508] Master 127.16.187.190:39141 requested a full tablet report, sending...
I20261117 09:06:07.480451 17173 ts_manager.cc:210] Registered new tserver with Master: 5d8ecddcda974b718d9f5360545a674c (127.16.187.131:43889)
I20261117 09:06:07.480835 17173 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.131:34043
I20261117 09:06:07.481359 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 17497
I20261117 09:06:07.481448 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-2/wal/instance
I20261117 09:06:07.482721 17134 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20261117 09:06:07.494598 17173 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32810:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20261117 09:06:07.494875 17173 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261117 09:06:07.500890 17296 tablet_service.cc:1500] Processing CreateTablet for tablet a80f17a22e6e4e67850cf1af04862814 (DEFAULT_TABLE table=test-workload [id=73f934cd4c5a41f68d6315602dd5b153]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:07.501049 17428 tablet_service.cc:1500] Processing CreateTablet for tablet a80f17a22e6e4e67850cf1af04862814 (DEFAULT_TABLE table=test-workload [id=73f934cd4c5a41f68d6315602dd5b153]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:07.501194 17296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80f17a22e6e4e67850cf1af04862814. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:07.501260 17428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80f17a22e6e4e67850cf1af04862814. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:07.502022 17560 tablet_service.cc:1500] Processing CreateTablet for tablet a80f17a22e6e4e67850cf1af04862814 (DEFAULT_TABLE table=test-workload [id=73f934cd4c5a41f68d6315602dd5b153]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:07.502272 17560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80f17a22e6e4e67850cf1af04862814. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:07.503818 17645 tablet_bootstrap.cc:492] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Bootstrap starting.
I20261117 09:06:07.504019 17646 tablet_bootstrap.cc:492] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7: Bootstrap starting.
I20261117 09:06:07.504256 17647 tablet_bootstrap.cc:492] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c: Bootstrap starting.
I20261117 09:06:07.504639 17646 tablet_bootstrap.cc:654] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:07.504889 17647 tablet_bootstrap.cc:654] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:07.504889 17646 log.cc:826] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:07.505162 17647 log.cc:826] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:07.505452 17646 tablet_bootstrap.cc:492] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7: No bootstrap required, opened a new log
I20261117 09:06:07.505506 17646 ts_tablet_manager.cc:1449] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:07.505560 17645 tablet_bootstrap.cc:654] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:07.505793 17645 log.cc:826] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:07.505887 17647 tablet_bootstrap.cc:492] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c: No bootstrap required, opened a new log
I20261117 09:06:07.505947 17647 ts_tablet_manager.cc:1449] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:07.506295 17645 tablet_bootstrap.cc:492] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: No bootstrap required, opened a new log
I20261117 09:06:07.506350 17645 ts_tablet_manager.cc:1449] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20261117 09:06:07.507001 17646 raft_consensus.cc:360] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.507138 17646 raft_consensus.cc:386] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:07.507165 17646 raft_consensus.cc:748] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4291cda32014c93b90c840294450aa7, State: Initialized, Role: FOLLOWER
I20261117 09:06:07.507246 17646 consensus_queue.cc:261] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.507469 17646 ts_tablet_manager.cc:1480] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:07.507506 17494 heartbeater.cc:500] Master 127.16.187.190:39141 was elected leader, sending a full tablet report...
I20261117 09:06:07.507498 17647 raft_consensus.cc:360] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.507599 17647 raft_consensus.cc:386] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:07.507622 17647 raft_consensus.cc:748] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8ecddcda974b718d9f5360545a674c, State: Initialized, Role: FOLLOWER
I20261117 09:06:07.507635 17645 raft_consensus.cc:360] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.507696 17647 consensus_queue.cc:261] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.507756 17645 raft_consensus.cc:386] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:07.507789 17645 raft_consensus.cc:748] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acbcb0f25f4848eaa9756e2eb85a97bd, State: Initialized, Role: FOLLOWER
I20261117 09:06:07.507891 17647 ts_tablet_manager.cc:1480] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:07.507879 17645 consensus_queue.cc:261] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.507988 17626 heartbeater.cc:500] Master 127.16.187.190:39141 was elected leader, sending a full tablet report...
I20261117 09:06:07.508083 17645 ts_tablet_manager.cc:1480] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261117 09:06:07.508123 17362 heartbeater.cc:500] Master 127.16.187.190:39141 was elected leader, sending a full tablet report...
W20261117 09:06:07.604497 17495 tablet.cc:2441] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261117 09:06:07.726073 17627 tablet.cc:2441] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261117 09:06:07.733394 17363 tablet.cc:2441] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261117 09:06:07.790280 17652 raft_consensus.cc:499] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261117 09:06:07.790390 17652 raft_consensus.cc:521] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.790705 17652 leader_election.cc:290] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129:45937), d4291cda32014c93b90c840294450aa7 (127.16.187.130:38939)
I20261117 09:06:07.793561 17448 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a80f17a22e6e4e67850cf1af04862814" candidate_uuid: "5d8ecddcda974b718d9f5360545a674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4291cda32014c93b90c840294450aa7" is_pre_election: true
I20261117 09:06:07.793732 17448 raft_consensus.cc:2490] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d8ecddcda974b718d9f5360545a674c in term 0.
I20261117 09:06:07.793946 17512 leader_election.cc:304] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d8ecddcda974b718d9f5360545a674c, d4291cda32014c93b90c840294450aa7; no voters:
I20261117 09:06:07.794011 17316 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a80f17a22e6e4e67850cf1af04862814" candidate_uuid: "5d8ecddcda974b718d9f5360545a674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" is_pre_election: true
I20261117 09:06:07.794133 17316 raft_consensus.cc:2490] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d8ecddcda974b718d9f5360545a674c in term 0.
I20261117 09:06:07.794198 17652 raft_consensus.cc:2826] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261117 09:06:07.794251 17652 raft_consensus.cc:499] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261117 09:06:07.794281 17652 raft_consensus.cc:3096] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:07.794952 17652 raft_consensus.cc:521] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.795085 17652 leader_election.cc:290] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [CANDIDATE]: Term 1 election: Requested vote from peers acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129:45937), d4291cda32014c93b90c840294450aa7 (127.16.187.130:38939)
I20261117 09:06:07.795279 17316 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a80f17a22e6e4e67850cf1af04862814" candidate_uuid: "5d8ecddcda974b718d9f5360545a674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd"
I20261117 09:06:07.795302 17448 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a80f17a22e6e4e67850cf1af04862814" candidate_uuid: "5d8ecddcda974b718d9f5360545a674c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4291cda32014c93b90c840294450aa7"
I20261117 09:06:07.795356 17316 raft_consensus.cc:3096] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:07.795377 17448 raft_consensus.cc:3096] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:07.795955 17316 raft_consensus.cc:2490] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d8ecddcda974b718d9f5360545a674c in term 1.
I20261117 09:06:07.796129 17513 leader_election.cc:304] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d8ecddcda974b718d9f5360545a674c, acbcb0f25f4848eaa9756e2eb85a97bd; no voters:
I20261117 09:06:07.796222 17652 raft_consensus.cc:2826] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:07.796293 17448 raft_consensus.cc:2490] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d8ecddcda974b718d9f5360545a674c in term 1.
I20261117 09:06:07.796373 17652 raft_consensus.cc:705] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 1 LEADER]: Becoming Leader. State: Replica: 5d8ecddcda974b718d9f5360545a674c, State: Running, Role: LEADER
I20261117 09:06:07.796471 17652 consensus_queue.cc:238] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.797417 17657 consensus_queue.cc:1049] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:07.797590 17657 consensus_queue.cc:1049] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:07.797506 17172 catalog_manager.cc:5840] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d8ecddcda974b718d9f5360545a674c (127.16.187.131). New cstate: current_term: 1 leader_uuid: "5d8ecddcda974b718d9f5360545a674c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 45937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } health_report { overall_health: UNKNOWN } } }
W20261117 09:06:07.894562 17356 generic_service.cc:139] {username='slave'} at 127.0.0.1:50854 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261117 09:06:07.894649 17356 generic_service.cc:175] {username='slave'} at 127.0.0.1:50854 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'false'
I20261117 09:06:07.895025 17356 generic_service.cc:175] {username='slave'} at 127.0.0.1:50854 changed flags via RPC: heartbeat_inject_latency_before_heartbeat_ms from '0' to '5000'
W20261117 09:06:07.895433 17488 generic_service.cc:139] {username='slave'} at 127.0.0.1:38770 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261117 09:06:07.895491 17488 generic_service.cc:175] {username='slave'} at 127.0.0.1:38770 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'false'
I20261117 09:06:07.895807 17488 generic_service.cc:175] {username='slave'} at 127.0.0.1:38770 changed flags via RPC: heartbeat_inject_latency_before_heartbeat_ms from '0' to '5000'
W20261117 09:06:07.896179 17620 generic_service.cc:139] {username='slave'} at 127.0.0.1:47528 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261117 09:06:07.896236 17620 generic_service.cc:175] {username='slave'} at 127.0.0.1:47528 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'false'
I20261117 09:06:07.896512 17620 generic_service.cc:175] {username='slave'} at 127.0.0.1:47528 changed flags via RPC: heartbeat_inject_latency_before_heartbeat_ms from '0' to '2000'
I20261117 09:06:07.897073 17580 consensus_queue.cc:238] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.897477 17448 raft_consensus.cc:1285] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 1 FOLLOWER]: Refusing update from remote peer 5d8ecddcda974b718d9f5360545a674c: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20261117 09:06:07.897665 17655 consensus_queue.cc:1049] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20261117 09:06:07.898316 17658 raft_consensus.cc:2983] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 1 LEADER]: Committing config change with OpId 1.33: config changed from index -1 to 33, VOTER acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } } }
I20261117 09:06:07.898422 17448 raft_consensus.cc:2983] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index -1 to 33, VOTER acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } } }
W20261117 09:06:07.899647 17620 generic_service.cc:139] {username='slave'} at 127.0.0.1:47528 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261117 09:06:07.899693 17620 generic_service.cc:175] {username='slave'} at 127.0.0.1:47528 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'true'
I20261117 09:06:07.900054 17448 tablet_service.cc:2033] Received Run Leader Election RPC: tablet_id: "a80f17a22e6e4e67850cf1af04862814"
dest_uuid: "d4291cda32014c93b90c840294450aa7"
from {username='slave'} at 127.0.0.1:38770
I20261117 09:06:07.900117 17448 raft_consensus.cc:499] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20261117 09:06:07.900137 17448 raft_consensus.cc:3096] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 1 FOLLOWER]: Advancing to term 2
I20261117 09:06:07.900560 17448 raft_consensus.cc:521] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.900730 17448 leader_election.cc:290] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [CANDIDATE]: Term 2 election: Requested vote from peers 5d8ecddcda974b718d9f5360545a674c (127.16.187.131:43889)
I20261117 09:06:07.903461 17580 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a80f17a22e6e4e67850cf1af04862814" candidate_uuid: "d4291cda32014c93b90c840294450aa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 33 } } ignore_live_leader: true dest_uuid: "5d8ecddcda974b718d9f5360545a674c"
I20261117 09:06:07.903558 17580 raft_consensus.cc:3091] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 1 LEADER]: Stepping down as leader of term 1
I20261117 09:06:07.903584 17580 raft_consensus.cc:748] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5d8ecddcda974b718d9f5360545a674c, State: Running, Role: LEADER
I20261117 09:06:07.903632 17580 consensus_queue.cc:261] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
I20261117 09:06:07.903698 17580 raft_consensus.cc:3096] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 1 FOLLOWER]: Advancing to term 2
I20261117 09:06:07.904309 17580 raft_consensus.cc:2490] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4291cda32014c93b90c840294450aa7 in term 2.
I20261117 09:06:07.904718 17381 leader_election.cc:304] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 5d8ecddcda974b718d9f5360545a674c, d4291cda32014c93b90c840294450aa7; no voters:
I20261117 09:06:07.904858 17651 raft_consensus.cc:2826] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 2 FOLLOWER]: Leader election won for term 2
I20261117 09:06:07.905030 17651 raft_consensus.cc:705] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 2 LEADER]: Becoming Leader. State: Replica: d4291cda32014c93b90c840294450aa7, State: Running, Role: LEADER
I20261117 09:06:07.905117 17651 consensus_queue.cc:238] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } }
W20261117 09:06:07.905838 17381 consensus_peers.cc:597] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 -> Peer 5d8ecddcda974b718d9f5360545a674c (127.16.187.131:43889): Couldn't send request to peer 5d8ecddcda974b718d9f5360545a674c. Error code: UNKNOWN_ERROR (1). Status: Illegal state: Rejected: --follower_reject_update_consensus_requests is set to true.. This is attempt 1: this message will repeat every 5th retry.
I20261117 09:06:09.900059 17171 catalog_manager.cc:5840] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c reported cstate change: config changed from index -1 to 33, term changed from 1 to 2, leader changed from 5d8ecddcda974b718d9f5360545a674c (127.16.187.131) to <none>, VOTER acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129) evicted. New cstate: current_term: 2 committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } } }
I20261117 09:06:09.901561 17296 tablet_service.cc:1547] Processing DeleteTablet for tablet a80f17a22e6e4e67850cf1af04862814 with delete_type TABLET_DATA_TOMBSTONED (TS acbcb0f25f4848eaa9756e2eb85a97bd not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:50838
I20261117 09:06:09.901785 17687 tablet_replica.cc:333] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: stopping tablet replica
I20261117 09:06:09.901865 17687 raft_consensus.cc:2261] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20261117 09:06:09.901964 17687 raft_consensus.cc:2290] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20261117 09:06:09.902283 17687 ts_tablet_manager.cc:1962] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20261117 09:06:09.903891 17687 ts_tablet_manager.cc:1975] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.32
I20261117 09:06:09.903990 17687 log.cc:1199] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Deleting WAL directory at /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794906366961029-17134-0/minicluster-data/ts-0/wal/wals/a80f17a22e6e4e67850cf1af04862814
I20261117 09:06:09.904404 17158 catalog_manager.cc:5065] TS acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129:45937): tablet a80f17a22e6e4e67850cf1af04862814 (table test-workload [id=73f934cd4c5a41f68d6315602dd5b153]) successfully deleted
W20261117 09:06:09.932885 17620 generic_service.cc:139] {username='slave'} at 127.0.0.1:47528 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261117 09:06:09.932966 17620 generic_service.cc:175] {username='slave'} at 127.0.0.1:47528 changed flags via RPC: follower_reject_update_consensus_requests from 'true' to 'false'
I20261117 09:06:09.934605 17134 cluster_itest_util.cc:258] Not converged past 1 yet: 2.34 1.33 <uninitialized op>
I20261117 09:06:10.036348 17134 cluster_itest_util.cc:258] Not converged past 1 yet: 2.34 1.33 <uninitialized op>
I20261117 09:06:10.237653 17134 cluster_itest_util.cc:258] Not converged past 1 yet: 2.34 1.33 <uninitialized op>
I20261117 09:06:10.538977 17134 cluster_itest_util.cc:258] Not converged past 1 yet: 2.34 2.34 <uninitialized op>
I20261117 09:06:10.940341 17134 cluster_itest_util.cc:258] Not converged past 1 yet: 2.34 2.34 <uninitialized op>
I20261117 09:06:11.441594 17134 cluster_itest_util.cc:258] Not converged past 1 yet: 2.34 2.34 <uninitialized op>
I20261117 09:06:11.902171 17171 catalog_manager.cc:5840] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c reported cstate change: leader changed from <none> to d4291cda32014c93b90c840294450aa7 (127.16.187.130). New cstate: current_term: 2 leader_uuid: "d4291cda32014c93b90c840294450aa7" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } } }
I20261117 09:06:12.042876 17134 cluster_itest_util.cc:258] Not converged past 1 yet: 2.34 2.34 <uninitialized op>
I20261117 09:06:12.744480 17134 cluster_itest_util.cc:258] Not converged past 1 yet: 2.34 2.34 <uninitialized op>
I20261117 09:06:12.900645 17448 consensus_queue.cc:238] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 33, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 45937 } attrs { promote: true } }
I20261117 09:06:12.901360 17580 raft_consensus.cc:1285] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 2 FOLLOWER]: Refusing update from remote peer d4291cda32014c93b90c840294450aa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20261117 09:06:12.901615 17701 consensus_queue.cc:1049] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20261117 09:06:12.902551 17700 raft_consensus.cc:2983] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 33 to 35, NON_VOTER acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129) added. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 45937 } attrs { promote: true } } }
I20261117 09:06:12.902678 17580 raft_consensus.cc:2983] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 33 to 35, NON_VOTER acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129) added. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 45937 } attrs { promote: true } } }
I20261117 09:06:12.903693 17157 catalog_manager.cc:5247] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a80f17a22e6e4e67850cf1af04862814 with cas_config_opid_index 33: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20261117 09:06:12.904338 17382 consensus_peers.cc:597] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7 -> Peer acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129:45937): Couldn't send request to peer acbcb0f25f4848eaa9756e2eb85a97bd. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20261117 09:06:13.546001 17134 cluster_itest_util.cc:258] Not converged past 1 yet: 2.35 2.35 <uninitialized op>
I20261117 09:06:13.904268 17171 catalog_manager.cc:5840] T a80f17a22e6e4e67850cf1af04862814 P 5d8ecddcda974b718d9f5360545a674c reported cstate change: config changed from index 33 to 35, NON_VOTER acbcb0f25f4848eaa9756e2eb85a97bd (127.16.187.129) added. New cstate: current_term: 2 leader_uuid: "d4291cda32014c93b90c840294450aa7" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 45937 } attrs { promote: true } } }
I20261117 09:06:14.447412 17134 cluster_itest_util.cc:258] Not converged past 1 yet: 2.35 2.35 <uninitialized op>
I20261117 09:06:15.188694 17712 ts_tablet_manager.cc:979] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Initiating tablet copy from peer d4291cda32014c93b90c840294450aa7 (127.16.187.130:38939)
I20261117 09:06:15.188997 17712 tablet_copy_client.cc:323] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: tablet copy: Beginning tablet copy session from remote peer at address 127.16.187.130:38939
I20261117 09:06:15.191875 17468 tablet_copy_service.cc:140] P d4291cda32014c93b90c840294450aa7: Received BeginTabletCopySession request for tablet a80f17a22e6e4e67850cf1af04862814 from peer acbcb0f25f4848eaa9756e2eb85a97bd ({username='slave'} at 127.16.187.129:37729)
I20261117 09:06:15.191953 17468 tablet_copy_service.cc:161] P d4291cda32014c93b90c840294450aa7: Beginning new tablet copy session on tablet a80f17a22e6e4e67850cf1af04862814 from peer acbcb0f25f4848eaa9756e2eb85a97bd at {username='slave'} at 127.16.187.129:37729: session id = acbcb0f25f4848eaa9756e2eb85a97bd-a80f17a22e6e4e67850cf1af04862814
I20261117 09:06:15.192448 17468 tablet_copy_source_session.cc:215] T a80f17a22e6e4e67850cf1af04862814 P d4291cda32014c93b90c840294450aa7: Tablet Copy: opened 0 blocks and 1 log segments
I20261117 09:06:15.192973 17712 ts_tablet_manager.cc:1962] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Deleting tablet data with delete state TABLET_DATA_COPYING
I20261117 09:06:15.194306 17712 ts_tablet_manager.cc:1975] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.32
I20261117 09:06:15.194411 17712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80f17a22e6e4e67850cf1af04862814. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:15.194988 17712 tablet_copy_client.cc:806] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: tablet copy: Starting download of 0 data blocks...
I20261117 09:06:15.195088 17712 tablet_copy_client.cc:670] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: tablet copy: Starting download of 1 WAL segments...
I20261117 09:06:15.196161 17712 tablet_copy_client.cc:538] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261117 09:06:15.196727 17712 tablet_bootstrap.cc:492] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Bootstrap starting.
I20261117 09:06:15.202625 17712 tablet_bootstrap.cc:492] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=35 overwritten=0 applied=35 ignored=0} inserts{seen=1550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261117 09:06:15.202826 17712 tablet_bootstrap.cc:492] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Bootstrap complete.
I20261117 09:06:15.202934 17712 ts_tablet_manager.cc:1449] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20261117 09:06:15.203073 17712 raft_consensus.cc:360] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 45937 } attrs { promote: true } }
I20261117 09:06:15.203132 17712 raft_consensus.cc:748] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: acbcb0f25f4848eaa9756e2eb85a97bd, State: Initialized, Role: LEARNER
I20261117 09:06:15.203167 17712 consensus_queue.cc:261] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 35, Last appended: 2.35, Last appended by leader: 35, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5d8ecddcda974b718d9f5360545a674c" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 43889 } } peers { permanent_uuid: "d4291cda32014c93b90c840294450aa7" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 38939 } } peers { permanent_uuid: "acbcb0f25f4848eaa9756e2eb85a97bd" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 45937 } attrs { promote: true } }
I20261117 09:06:15.203339 17712 ts_tablet_manager.cc:1480] T a80f17a22e6e4e67850cf1af04862814 P acbcb0f25f4848eaa9756e2eb85a97bd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:15.203646 17468 tablet_copy_service.cc:342] P d4291cda32014c93b90c840294450aa7: Request end of tablet copy session acbcb0f25f4848eaa9756e2eb85a97bd-a80f17a22e6e4e67850cf1af04862814 received from {username='slave'} at 127.16.187.129:37729
I20261117 09:06:15.203709 17468 tablet_copy_service.cc:434] P d4291cda32014c93b90c840294450aa7: ending tablet copy session acbcb0f25f4848eaa9756e2eb85a97bd-a80f17a22e6e4e67850cf1af04862814 on tablet a80f17a22e6e4e67850cf1af04862814 with peer acbcb0f25f4848eaa9756e2eb85a97bd
I20261117 09:06:15.449417 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 17233
I20261117 09:06:15.454025 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 17365
I20261117 09:06:15.458277 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 17497
I20261117 09:06:15.462327 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 17142
2026-11-17T09:06:15Z chronyd exiting
[ OK ] RaftConfigChangeITest.TestKudu2147 (8514 ms)
[ RUN ] RaftConfigChangeITest.TestNonVoterPromotion
2026-11-17T09:06:15Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-17T09:06:15Z Disabled control of system clock
I20261117 09:06:15.487722 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:36691
--webserver_interface=127.16.187.190
--webserver_port=0
--builtin_ntp_servers=127.16.187.148:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.187.190:36691
--raft_prepare_replacement_before_eviction=true
--catalog_manager_evict_excess_replicas=false with env {}
W20261117 09:06:15.563570 17721 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:15.563741 17721 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:15.563759 17721 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:15.565315 17721 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261117 09:06:15.565363 17721 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:15.565376 17721 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261117 09:06:15.565387 17721 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261117 09:06:15.567076 17721 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/wal
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.16.187.190:36691
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:36691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.187.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.17721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:15.567292 17721 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:15.567432 17721 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20261117 09:06:15.570099 17721 server_base.cc:1061] running on GCE node
W20261117 09:06:15.570118 17726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:15.570109 17727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:15.570109 17729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:15.570681 17721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:15.570900 17721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:15.572057 17721 hybrid_clock.cc:648] HybridClock initialized: now 1794906375572022 us; error 53 us; skew 500 ppm
I20261117 09:06:15.573236 17721 webserver.cc:577] Webserver started at http://127.16.187.190:37219/ using document root <none> and password file <none>
I20261117 09:06:15.573442 17721 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:15.573483 17721 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:15.573575 17721 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:15.574406 17721 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/data/instance:
uuid: "b5ed4de556404702948c024e387284e0"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.574739 17721 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/wal/instance:
uuid: "b5ed4de556404702948c024e387284e0"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.575901 17721 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20261117 09:06:15.576726 17735 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:15.576923 17721 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20261117 09:06:15.576995 17721 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/wal
uuid: "b5ed4de556404702948c024e387284e0"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.577059 17721 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:15.604207 17721 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:15.604480 17721 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:15.604588 17721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:15.608544 17721 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.190:36691
I20261117 09:06:15.608603 17787 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.190:36691 every 8 connection(s)
I20261117 09:06:15.608915 17721 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/data/info.pb
I20261117 09:06:15.609508 17788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:15.611789 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 17721
I20261117 09:06:15.611871 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/master-0/wal/instance
I20261117 09:06:15.612188 17788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0: Bootstrap starting.
I20261117 09:06:15.612990 17788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:15.613210 17788 log.cc:826] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:15.613754 17788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0: No bootstrap required, opened a new log
I20261117 09:06:15.614886 17788 raft_consensus.cc:360] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5ed4de556404702948c024e387284e0" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36691 } }
I20261117 09:06:15.615005 17788 raft_consensus.cc:386] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:15.615031 17788 raft_consensus.cc:748] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5ed4de556404702948c024e387284e0, State: Initialized, Role: FOLLOWER
I20261117 09:06:15.615162 17788 consensus_queue.cc:261] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5ed4de556404702948c024e387284e0" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36691 } }
I20261117 09:06:15.615237 17788 raft_consensus.cc:400] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261117 09:06:15.615268 17788 raft_consensus.cc:499] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261117 09:06:15.615301 17788 raft_consensus.cc:3096] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:15.615907 17788 raft_consensus.cc:521] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5ed4de556404702948c024e387284e0" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36691 } }
I20261117 09:06:15.616021 17788 leader_election.cc:304] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5ed4de556404702948c024e387284e0; no voters:
I20261117 09:06:15.616211 17788 leader_election.cc:290] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261117 09:06:15.616250 17793 raft_consensus.cc:2826] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:15.616385 17793 raft_consensus.cc:705] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [term 1 LEADER]: Becoming Leader. State: Replica: b5ed4de556404702948c024e387284e0, State: Running, Role: LEADER
I20261117 09:06:15.616446 17788 sys_catalog.cc:565] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [sys.catalog]: configured and running, proceeding with master startup.
I20261117 09:06:15.616485 17793 consensus_queue.cc:238] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5ed4de556404702948c024e387284e0" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36691 } }
I20261117 09:06:15.616875 17794 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5ed4de556404702948c024e387284e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5ed4de556404702948c024e387284e0" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36691 } } }
I20261117 09:06:15.616890 17795 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5ed4de556404702948c024e387284e0. Latest consensus state: current_term: 1 leader_uuid: "b5ed4de556404702948c024e387284e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5ed4de556404702948c024e387284e0" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36691 } } }
I20261117 09:06:15.616967 17794 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:15.616971 17795 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0 [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:15.617313 17801 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261117 09:06:15.617789 17801 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261117 09:06:15.619314 17801 catalog_manager.cc:1398] Generated new cluster ID: f83ffebeac7e4eb98e3d169b7ca7a646
I20261117 09:06:15.619370 17801 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261117 09:06:15.636946 17801 catalog_manager.cc:1421] Generated new certificate authority record
I20261117 09:06:15.637660 17801 catalog_manager.cc:1555] Loading token signing keys...
I20261117 09:06:15.648763 17801 catalog_manager.cc:6221] T 00000000000000000000000000000000 P b5ed4de556404702948c024e387284e0: Generated new TSK 0
I20261117 09:06:15.649040 17801 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261117 09:06:15.658499 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.129:0
--local_ip_for_outbound_sockets=127.16.187.129
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36691
--builtin_ntp_servers=127.16.187.148:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20261117 09:06:15.736440 17812 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:15.736678 17812 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:15.736730 17812 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:15.738621 17812 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:15.738686 17812 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.129
I20261117 09:06:15.740286 17812 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36691
--never_fsync=true
--heap_profile_path=/tmp/kudu.17812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:15.740476 17812 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:15.740631 17812 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:15.742859 17817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:15.742977 17812 server_base.cc:1061] running on GCE node
W20261117 09:06:15.742998 17820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:15.742861 17818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:15.743332 17812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:15.743517 17812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:15.744673 17812 hybrid_clock.cc:648] HybridClock initialized: now 1794906375744658 us; error 33 us; skew 500 ppm
I20261117 09:06:15.745666 17812 webserver.cc:577] Webserver started at http://127.16.187.129:42525/ using document root <none> and password file <none>
I20261117 09:06:15.745838 17812 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:15.745877 17812 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:15.745978 17812 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:15.746817 17812 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/data/instance:
uuid: "dca65d99f78b412997a17833c6642534"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.747105 17812 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/wal/instance:
uuid: "dca65d99f78b412997a17833c6642534"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.748178 17812 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:15.748826 17826 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:15.748982 17812 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:15.749037 17812 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/wal
uuid: "dca65d99f78b412997a17833c6642534"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.749091 17812 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:15.758105 17812 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:15.758363 17812 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:15.758478 17812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:15.758685 17812 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:15.758991 17812 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:15.759021 17812 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:15.759048 17812 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:15.759071 17812 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:15.764654 17812 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.129:32917
I20261117 09:06:15.764711 17939 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.129:32917 every 8 connection(s)
I20261117 09:06:15.765004 17812 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/data/info.pb
I20261117 09:06:15.769294 17940 heartbeater.cc:345] Connected to a master server at 127.16.187.190:36691
I20261117 09:06:15.769390 17940 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:15.769569 17940 heartbeater.cc:508] Master 127.16.187.190:36691 requested a full tablet report, sending...
I20261117 09:06:15.770095 17752 ts_manager.cc:210] Registered new tserver with Master: dca65d99f78b412997a17833c6642534 (127.16.187.129:32917)
I20261117 09:06:15.771049 17752 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.129:57789
I20261117 09:06:15.773324 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 17812
I20261117 09:06:15.773418 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-0/wal/instance
I20261117 09:06:15.774461 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.130:0
--local_ip_for_outbound_sockets=127.16.187.130
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36691
--builtin_ntp_servers=127.16.187.148:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20261117 09:06:15.852828 17943 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:15.852969 17943 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:15.852988 17943 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:15.854540 17943 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:15.854596 17943 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.130
I20261117 09:06:15.856150 17943 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36691
--never_fsync=true
--heap_profile_path=/tmp/kudu.17943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:15.856331 17943 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:15.856505 17943 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:15.858741 17948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:15.858835 17943 server_base.cc:1061] running on GCE node
W20261117 09:06:15.858764 17949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:15.858856 17951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:15.859108 17943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:15.859290 17943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:15.860421 17943 hybrid_clock.cc:648] HybridClock initialized: now 1794906375860407 us; error 31 us; skew 500 ppm
I20261117 09:06:15.861348 17943 webserver.cc:577] Webserver started at http://127.16.187.130:41363/ using document root <none> and password file <none>
I20261117 09:06:15.861531 17943 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:15.861567 17943 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:15.861655 17943 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:15.862421 17943 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/data/instance:
uuid: "16d0e4b1bc8e4606bb544ebdfabad11c"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.862787 17943 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/wal/instance:
uuid: "16d0e4b1bc8e4606bb544ebdfabad11c"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.863790 17943 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:15.864351 17957 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:15.864526 17943 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:15.864583 17943 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/wal
uuid: "16d0e4b1bc8e4606bb544ebdfabad11c"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.864636 17943 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:15.876937 17943 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:15.877192 17943 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:15.877300 17943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:15.877488 17943 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:15.877800 17943 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:15.877831 17943 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:15.877861 17943 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:15.877885 17943 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:15.883314 17943 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.130:42587
I20261117 09:06:15.883358 18070 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.130:42587 every 8 connection(s)
I20261117 09:06:15.883674 17943 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/data/info.pb
I20261117 09:06:15.887884 18071 heartbeater.cc:345] Connected to a master server at 127.16.187.190:36691
I20261117 09:06:15.888006 18071 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:15.888190 18071 heartbeater.cc:508] Master 127.16.187.190:36691 requested a full tablet report, sending...
I20261117 09:06:15.888563 17752 ts_manager.cc:210] Registered new tserver with Master: 16d0e4b1bc8e4606bb544ebdfabad11c (127.16.187.130:42587)
I20261117 09:06:15.888682 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 17943
I20261117 09:06:15.888767 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-1/wal/instance
I20261117 09:06:15.889014 17752 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.130:55757
I20261117 09:06:15.889739 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.131:0
--local_ip_for_outbound_sockets=127.16.187.131
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36691
--builtin_ntp_servers=127.16.187.148:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20261117 09:06:15.966763 18074 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:15.966909 18074 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:15.966926 18074 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:15.968535 18074 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:15.968595 18074 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.131
I20261117 09:06:15.970170 18074 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36691
--never_fsync=true
--heap_profile_path=/tmp/kudu.18074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:15.970347 18074 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:15.970507 18074 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:15.972730 18079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:15.972798 18074 server_base.cc:1061] running on GCE node
W20261117 09:06:15.972765 18082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:15.972740 18080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:15.973097 18074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:15.973266 18074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:15.974448 18074 hybrid_clock.cc:648] HybridClock initialized: now 1794906375974428 us; error 29 us; skew 500 ppm
I20261117 09:06:15.975451 18074 webserver.cc:577] Webserver started at http://127.16.187.131:40219/ using document root <none> and password file <none>
I20261117 09:06:15.975627 18074 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:15.975665 18074 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:15.975752 18074 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:15.976534 18074 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/data/instance:
uuid: "91dee767c56e4f819918ccd69ff12cd8"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.976835 18074 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/wal/instance:
uuid: "91dee767c56e4f819918ccd69ff12cd8"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.977999 18074 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:15.978729 18088 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:15.978871 18074 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:15.978971 18074 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/wal
uuid: "91dee767c56e4f819918ccd69ff12cd8"
format_stamp: "Formatted at 2026-11-17 09:06:15 on dist-test-slave-x52s"
I20261117 09:06:15.979027 18074 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:15.992326 18074 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:15.992594 18074 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:15.992702 18074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:15.992894 18074 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:15.993214 18074 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:15.993244 18074 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:15.993274 18074 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:15.993297 18074 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:15.998515 18074 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.131:33615
I20261117 09:06:15.998605 18201 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.131:33615 every 8 connection(s)
I20261117 09:06:15.998900 18074 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/data/info.pb
I20261117 09:06:16.002820 18202 heartbeater.cc:345] Connected to a master server at 127.16.187.190:36691
I20261117 09:06:16.002915 18202 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:16.003118 18202 heartbeater.cc:508] Master 127.16.187.190:36691 requested a full tablet report, sending...
I20261117 09:06:16.003458 17752 ts_manager.cc:210] Registered new tserver with Master: 91dee767c56e4f819918ccd69ff12cd8 (127.16.187.131:33615)
I20261117 09:06:16.003836 17752 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.131:41443
I20261117 09:06:16.003930 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 18074
I20261117 09:06:16.004011 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-2/wal/instance
I20261117 09:06:16.005097 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.132:0
--local_ip_for_outbound_sockets=127.16.187.132
--webserver_interface=127.16.187.132
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36691
--builtin_ntp_servers=127.16.187.148:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20261117 09:06:16.079963 18205 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:16.080098 18205 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:16.080124 18205 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:16.081647 18205 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:16.081702 18205 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.132
I20261117 09:06:16.083338 18205 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.16.187.132
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36691
--never_fsync=true
--heap_profile_path=/tmp/kudu.18205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:16.083513 18205 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:16.083678 18205 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:16.085954 18211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:16.085996 18213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:16.085933 18210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:16.086199 18205 server_base.cc:1061] running on GCE node
I20261117 09:06:16.086344 18205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:16.086516 18205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:16.087694 18205 hybrid_clock.cc:648] HybridClock initialized: now 1794906376087676 us; error 35 us; skew 500 ppm
I20261117 09:06:16.088675 18205 webserver.cc:577] Webserver started at http://127.16.187.132:43181/ using document root <none> and password file <none>
I20261117 09:06:16.088891 18205 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:16.088935 18205 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:16.089030 18205 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:16.089810 18205 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/data/instance:
uuid: "d4fe0daa887f4ebe966dc588ee0497c2"
format_stamp: "Formatted at 2026-11-17 09:06:16 on dist-test-slave-x52s"
I20261117 09:06:16.090132 18205 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/wal/instance:
uuid: "d4fe0daa887f4ebe966dc588ee0497c2"
format_stamp: "Formatted at 2026-11-17 09:06:16 on dist-test-slave-x52s"
I20261117 09:06:16.091279 18205 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261117 09:06:16.091919 18219 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:16.092075 18205 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261117 09:06:16.092137 18205 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/wal
uuid: "d4fe0daa887f4ebe966dc588ee0497c2"
format_stamp: "Formatted at 2026-11-17 09:06:16 on dist-test-slave-x52s"
I20261117 09:06:16.092195 18205 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:16.106952 18205 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:16.107218 18205 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:16.107339 18205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:16.107549 18205 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:16.107872 18205 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:16.107908 18205 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:16.107941 18205 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:16.107965 18205 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:16.113893 18205 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.132:39647
I20261117 09:06:16.113955 18332 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.132:39647 every 8 connection(s)
I20261117 09:06:16.114244 18205 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/data/info.pb
I20261117 09:06:16.118425 18333 heartbeater.cc:345] Connected to a master server at 127.16.187.190:36691
I20261117 09:06:16.118511 18333 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:16.118708 18333 heartbeater.cc:508] Master 127.16.187.190:36691 requested a full tablet report, sending...
I20261117 09:06:16.119063 17752 ts_manager.cc:210] Registered new tserver with Master: d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132:39647)
I20261117 09:06:16.119179 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 18205
I20261117 09:06:16.119256 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794906366961029-17134-0/minicluster-data/ts-3/wal/instance
I20261117 09:06:16.119472 17752 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.132:34369
I20261117 09:06:16.120170 17134 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261117 09:06:16.130335 17752 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60254:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261117 09:06:16.136868 18005 tablet_service.cc:1500] Processing CreateTablet for tablet 843b7ae6b2b34a968a46cd13d604f2b6 (DEFAULT_TABLE table=test-workload [id=72ba6c08559b401089965d09b00cb5d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:16.137169 18005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 843b7ae6b2b34a968a46cd13d604f2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:16.137432 18136 tablet_service.cc:1500] Processing CreateTablet for tablet 843b7ae6b2b34a968a46cd13d604f2b6 (DEFAULT_TABLE table=test-workload [id=72ba6c08559b401089965d09b00cb5d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:16.137663 18136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 843b7ae6b2b34a968a46cd13d604f2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:16.139281 18352 tablet_bootstrap.cc:492] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c: Bootstrap starting.
I20261117 09:06:16.139880 18353 tablet_bootstrap.cc:492] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8: Bootstrap starting.
I20261117 09:06:16.139891 17874 tablet_service.cc:1500] Processing CreateTablet for tablet 843b7ae6b2b34a968a46cd13d604f2b6 (DEFAULT_TABLE table=test-workload [id=72ba6c08559b401089965d09b00cb5d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:16.140112 18352 tablet_bootstrap.cc:654] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:16.140141 17874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 843b7ae6b2b34a968a46cd13d604f2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:16.140373 18352 log.cc:826] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:16.140487 18353 tablet_bootstrap.cc:654] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:16.140928 18353 log.cc:826] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:16.140973 18352 tablet_bootstrap.cc:492] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c: No bootstrap required, opened a new log
I20261117 09:06:16.141026 18352 ts_tablet_manager.cc:1449] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:16.141553 18353 tablet_bootstrap.cc:492] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8: No bootstrap required, opened a new log
I20261117 09:06:16.141602 18353 ts_tablet_manager.cc:1449] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:16.142201 18356 tablet_bootstrap.cc:492] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534: Bootstrap starting.
I20261117 09:06:16.142488 18352 raft_consensus.cc:360] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } }
I20261117 09:06:16.142678 18352 raft_consensus.cc:386] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:16.142709 18352 raft_consensus.cc:748] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d0e4b1bc8e4606bb544ebdfabad11c, State: Initialized, Role: FOLLOWER
I20261117 09:06:16.142792 18356 tablet_bootstrap.cc:654] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:16.142791 18352 consensus_queue.cc:261] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } }
I20261117 09:06:16.143023 18352 ts_tablet_manager.cc:1480] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:16.143062 18071 heartbeater.cc:500] Master 127.16.187.190:36691 was elected leader, sending a full tablet report...
I20261117 09:06:16.143110 18353 raft_consensus.cc:360] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } }
I20261117 09:06:16.143221 18353 raft_consensus.cc:386] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:16.143244 18353 raft_consensus.cc:748] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91dee767c56e4f819918ccd69ff12cd8, State: Initialized, Role: FOLLOWER
I20261117 09:06:16.143011 18356 log.cc:826] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:16.143325 18353 consensus_queue.cc:261] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } }
I20261117 09:06:16.143860 18353 ts_tablet_manager.cc:1480] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:16.143970 18202 heartbeater.cc:500] Master 127.16.187.190:36691 was elected leader, sending a full tablet report...
I20261117 09:06:16.144178 18356 tablet_bootstrap.cc:492] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534: No bootstrap required, opened a new log
I20261117 09:06:16.144287 18356 ts_tablet_manager.cc:1449] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:16.145372 18356 raft_consensus.cc:360] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } }
I20261117 09:06:16.145471 18356 raft_consensus.cc:386] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:16.145488 18356 raft_consensus.cc:748] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dca65d99f78b412997a17833c6642534, State: Initialized, Role: FOLLOWER
I20261117 09:06:16.145552 18356 consensus_queue.cc:261] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } }
I20261117 09:06:16.145756 18356 ts_tablet_manager.cc:1480] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:16.145798 17940 heartbeater.cc:500] Master 127.16.187.190:36691 was elected leader, sending a full tablet report...
W20261117 09:06:16.249362 18203 tablet.cc:2441] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261117 09:06:16.266119 17941 tablet.cc:2441] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261117 09:06:16.369913 18357 raft_consensus.cc:499] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261117 09:06:16.370044 18357 raft_consensus.cc:521] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } }
I20261117 09:06:16.370325 18357 leader_election.cc:290] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91dee767c56e4f819918ccd69ff12cd8 (127.16.187.131:33615), dca65d99f78b412997a17833c6642534 (127.16.187.129:32917)
I20261117 09:06:16.373049 18156 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "843b7ae6b2b34a968a46cd13d604f2b6" candidate_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91dee767c56e4f819918ccd69ff12cd8" is_pre_election: true
I20261117 09:06:16.373232 18156 raft_consensus.cc:2490] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16d0e4b1bc8e4606bb544ebdfabad11c in term 0.
I20261117 09:06:16.373453 17959 leader_election.cc:304] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16d0e4b1bc8e4606bb544ebdfabad11c, 91dee767c56e4f819918ccd69ff12cd8; no voters:
I20261117 09:06:16.373530 17894 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "843b7ae6b2b34a968a46cd13d604f2b6" candidate_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dca65d99f78b412997a17833c6642534" is_pre_election: true
I20261117 09:06:16.373572 18357 raft_consensus.cc:2826] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261117 09:06:16.373624 18357 raft_consensus.cc:499] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261117 09:06:16.373646 18357 raft_consensus.cc:3096] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:16.373672 17894 raft_consensus.cc:2490] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16d0e4b1bc8e4606bb544ebdfabad11c in term 0.
I20261117 09:06:16.374346 18357 raft_consensus.cc:521] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } }
I20261117 09:06:16.374473 18357 leader_election.cc:290] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [CANDIDATE]: Term 1 election: Requested vote from peers 91dee767c56e4f819918ccd69ff12cd8 (127.16.187.131:33615), dca65d99f78b412997a17833c6642534 (127.16.187.129:32917)
I20261117 09:06:16.374702 18156 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "843b7ae6b2b34a968a46cd13d604f2b6" candidate_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91dee767c56e4f819918ccd69ff12cd8"
I20261117 09:06:16.374723 17894 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "843b7ae6b2b34a968a46cd13d604f2b6" candidate_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dca65d99f78b412997a17833c6642534"
I20261117 09:06:16.374781 18156 raft_consensus.cc:3096] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:16.374789 17894 raft_consensus.cc:3096] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:16.375403 17894 raft_consensus.cc:2490] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16d0e4b1bc8e4606bb544ebdfabad11c in term 1.
I20261117 09:06:16.375569 17959 leader_election.cc:304] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16d0e4b1bc8e4606bb544ebdfabad11c, dca65d99f78b412997a17833c6642534; no voters:
I20261117 09:06:16.375663 18357 raft_consensus.cc:2826] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:16.375691 18156 raft_consensus.cc:2490] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16d0e4b1bc8e4606bb544ebdfabad11c in term 1.
I20261117 09:06:16.375809 18357 raft_consensus.cc:705] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 1 LEADER]: Becoming Leader. State: Replica: 16d0e4b1bc8e4606bb544ebdfabad11c, State: Running, Role: LEADER
I20261117 09:06:16.375905 18357 consensus_queue.cc:238] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } }
I20261117 09:06:16.376837 17751 catalog_manager.cc:5840] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c reported cstate change: term changed from 0 to 1, leader changed from <none> to 16d0e4b1bc8e4606bb544ebdfabad11c (127.16.187.130). New cstate: current_term: 1 leader_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } health_report { overall_health: UNKNOWN } } }
I20261117 09:06:16.376917 18362 consensus_queue.cc:1049] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [LEADER]: Connected to new peer: Peer: permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:16.377008 18365 consensus_queue.cc:1049] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:16.420382 18025 consensus_queue.cc:238] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: NON_VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: true } }
I20261117 09:06:16.421245 17894 raft_consensus.cc:1285] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [term 1 FOLLOWER]: Refusing update from remote peer 16d0e4b1bc8e4606bb544ebdfabad11c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20261117 09:06:16.421253 18156 raft_consensus.cc:1285] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [term 1 FOLLOWER]: Refusing update from remote peer 16d0e4b1bc8e4606bb544ebdfabad11c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20261117 09:06:16.421499 18357 consensus_queue.cc:1049] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [LEADER]: Connected to new peer: Peer: permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20261117 09:06:16.421679 18364 consensus_queue.cc:1049] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20261117 09:06:16.422448 18357 raft_consensus.cc:2983] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: NON_VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: true } } }
I20261117 09:06:16.423707 17894 raft_consensus.cc:2983] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: NON_VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: true } } }
I20261117 09:06:16.423792 17749 catalog_manager.cc:5840] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c reported cstate change: config changed from index -1 to 2, NON_VOTER d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132) added. New cstate: current_term: 1 leader_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: NON_VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20261117 09:06:16.424728 18156 raft_consensus.cc:2983] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: NON_VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: true } } }
W20261117 09:06:16.425992 17960 consensus_peers.cc:597] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c -> Peer d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132:39647): Couldn't send request to peer d4fe0daa887f4ebe966dc588ee0497c2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 843b7ae6b2b34a968a46cd13d604f2b6. This is attempt 1: this message will repeat every 5th retry.
I20261117 09:06:16.961992 18373 ts_tablet_manager.cc:979] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: Initiating tablet copy from peer 16d0e4b1bc8e4606bb544ebdfabad11c (127.16.187.130:42587)
I20261117 09:06:16.962208 18373 tablet_copy_client.cc:323] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: tablet copy: Beginning tablet copy session from remote peer at address 127.16.187.130:42587
I20261117 09:06:16.964897 18045 tablet_copy_service.cc:140] P 16d0e4b1bc8e4606bb544ebdfabad11c: Received BeginTabletCopySession request for tablet 843b7ae6b2b34a968a46cd13d604f2b6 from peer d4fe0daa887f4ebe966dc588ee0497c2 ({username='slave'} at 127.16.187.132:53779)
I20261117 09:06:16.964977 18045 tablet_copy_service.cc:161] P 16d0e4b1bc8e4606bb544ebdfabad11c: Beginning new tablet copy session on tablet 843b7ae6b2b34a968a46cd13d604f2b6 from peer d4fe0daa887f4ebe966dc588ee0497c2 at {username='slave'} at 127.16.187.132:53779: session id = d4fe0daa887f4ebe966dc588ee0497c2-843b7ae6b2b34a968a46cd13d604f2b6
I20261117 09:06:16.965536 18045 tablet_copy_source_session.cc:215] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c: Tablet Copy: opened 0 blocks and 1 log segments
I20261117 09:06:16.966351 18373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 843b7ae6b2b34a968a46cd13d604f2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:16.968231 18373 tablet_copy_client.cc:806] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: tablet copy: Starting download of 0 data blocks...
I20261117 09:06:16.968376 18373 tablet_copy_client.cc:670] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: tablet copy: Starting download of 1 WAL segments...
I20261117 09:06:16.969231 18373 tablet_copy_client.cc:538] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261117 09:06:16.969900 18373 tablet_bootstrap.cc:492] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: Bootstrap starting.
I20261117 09:06:16.970999 18373 log.cc:826] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:16.971926 18373 tablet_bootstrap.cc:492] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261117 09:06:16.972128 18373 tablet_bootstrap.cc:492] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: Bootstrap complete.
I20261117 09:06:16.972239 18373 ts_tablet_manager.cc:1449] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:16.973326 18373 raft_consensus.cc:360] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: NON_VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: true } }
I20261117 09:06:16.973460 18373 raft_consensus.cc:748] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d4fe0daa887f4ebe966dc588ee0497c2, State: Initialized, Role: LEARNER
I20261117 09:06:16.973526 18373 consensus_queue.cc:261] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: NON_VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: true } }
I20261117 09:06:16.973776 18373 ts_tablet_manager.cc:1480] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:16.973811 18333 heartbeater.cc:500] Master 127.16.187.190:36691 was elected leader, sending a full tablet report...
I20261117 09:06:16.974083 18045 tablet_copy_service.cc:342] P 16d0e4b1bc8e4606bb544ebdfabad11c: Request end of tablet copy session d4fe0daa887f4ebe966dc588ee0497c2-843b7ae6b2b34a968a46cd13d604f2b6 received from {username='slave'} at 127.16.187.132:53779
I20261117 09:06:16.974153 18045 tablet_copy_service.cc:434] P 16d0e4b1bc8e4606bb544ebdfabad11c: ending tablet copy session d4fe0daa887f4ebe966dc588ee0497c2-843b7ae6b2b34a968a46cd13d604f2b6 on tablet 843b7ae6b2b34a968a46cd13d604f2b6 with peer d4fe0daa887f4ebe966dc588ee0497c2
I20261117 09:06:17.374642 18287 raft_consensus.cc:1227] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20261117 09:06:17.767637 18357 raft_consensus.cc:1074] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c: attempting to promote NON_VOTER d4fe0daa887f4ebe966dc588ee0497c2 to VOTER
I20261117 09:06:17.767884 18357 consensus_queue.cc:238] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: false } }
I20261117 09:06:17.768492 18156 raft_consensus.cc:1285] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [term 1 FOLLOWER]: Refusing update from remote peer 16d0e4b1bc8e4606bb544ebdfabad11c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20261117 09:06:17.768482 17894 raft_consensus.cc:1285] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [term 1 FOLLOWER]: Refusing update from remote peer 16d0e4b1bc8e4606bb544ebdfabad11c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20261117 09:06:17.768522 18287 raft_consensus.cc:1285] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2 [term 1 LEARNER]: Refusing update from remote peer 16d0e4b1bc8e4606bb544ebdfabad11c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20261117 09:06:17.768710 18379 consensus_queue.cc:1049] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20261117 09:06:17.768836 18381 consensus_queue.cc:1049] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [LEADER]: Connected to new peer: Peer: permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20261117 09:06:17.768934 18382 consensus_queue.cc:1049] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20261117 09:06:17.769639 18379 raft_consensus.cc:2983] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: false } } }
I20261117 09:06:17.769773 18156 raft_consensus.cc:2983] T 843b7ae6b2b34a968a46cd13d604f2b6 P 91dee767c56e4f819918ccd69ff12cd8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: false } } }
I20261117 09:06:17.770031 18287 raft_consensus.cc:2983] T 843b7ae6b2b34a968a46cd13d604f2b6 P d4fe0daa887f4ebe966dc588ee0497c2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: false } } }
I20261117 09:06:17.770951 17749 catalog_manager.cc:5840] T 843b7ae6b2b34a968a46cd13d604f2b6 P 16d0e4b1bc8e4606bb544ebdfabad11c reported cstate change: config changed from index 2 to 3, d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20261117 09:06:17.771353 17894 raft_consensus.cc:2983] T 843b7ae6b2b34a968a46cd13d604f2b6 P dca65d99f78b412997a17833c6642534 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, d4fe0daa887f4ebe966dc588ee0497c2 (127.16.187.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "16d0e4b1bc8e4606bb544ebdfabad11c" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 42587 } } peers { permanent_uuid: "91dee767c56e4f819918ccd69ff12cd8" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 33615 } } peers { permanent_uuid: "dca65d99f78b412997a17833c6642534" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 32917 } } peers { permanent_uuid: "d4fe0daa887f4ebe966dc588ee0497c2" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 39647 } attrs { promote: false } } }
I20261117 09:06:17.845817 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 17812
I20261117 09:06:17.850993 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 17943
I20261117 09:06:17.855369 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 18074
I20261117 09:06:17.859465 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 18205
I20261117 09:06:17.863770 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 17721
2026-11-17T09:06:17Z chronyd exiting
[ OK ] RaftConfigChangeITest.TestNonVoterPromotion (2402 ms)
[ RUN ] RaftConfigChangeITest.TestBulkChangeConfig
2026-11-17T09:06:17Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-17T09:06:17Z Disabled control of system clock
I20261117 09:06:17.889402 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:38925
--webserver_interface=127.16.187.190
--webserver_port=0
--builtin_ntp_servers=127.16.187.148:36577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.187.190:38925
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261117 09:06:17.965724 18395 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:17.965888 18395 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:17.965905 18395 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:17.967513 18395 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261117 09:06:17.967559 18395 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:17.967572 18395 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261117 09:06:17.967583 18395 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261117 09:06:17.969182 18395 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:36577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.16.187.190:38925
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:38925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.187.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.18395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:17.969347 18395 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:17.969476 18395 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:17.971596 18401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:17.971611 18400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:17.971657 18395 server_base.cc:1061] running on GCE node
W20261117 09:06:17.971619 18403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:17.971994 18395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:17.972203 18395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:17.973320 18395 hybrid_clock.cc:648] HybridClock initialized: now 1794906377973309 us; error 26 us; skew 500 ppm
I20261117 09:06:17.974244 18395 webserver.cc:577] Webserver started at http://127.16.187.190:40519/ using document root <none> and password file <none>
I20261117 09:06:17.974406 18395 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:17.974442 18395 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:17.974550 18395 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:17.975342 18395 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/data/instance:
uuid: "516cfaada8ec4b4b9e6db46045a6cfb2"
format_stamp: "Formatted at 2026-11-17 09:06:17 on dist-test-slave-x52s"
I20261117 09:06:17.975644 18395 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/wal/instance:
uuid: "516cfaada8ec4b4b9e6db46045a6cfb2"
format_stamp: "Formatted at 2026-11-17 09:06:17 on dist-test-slave-x52s"
I20261117 09:06:17.976666 18395 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:17.977299 18409 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:17.977456 18395 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:17.977511 18395 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/wal
uuid: "516cfaada8ec4b4b9e6db46045a6cfb2"
format_stamp: "Formatted at 2026-11-17 09:06:17 on dist-test-slave-x52s"
I20261117 09:06:17.977566 18395 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:17.988672 18395 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:17.988942 18395 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:17.989044 18395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:17.992791 18395 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.190:38925
I20261117 09:06:17.992820 18461 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.190:38925 every 8 connection(s)
I20261117 09:06:17.993147 18395 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/data/info.pb
I20261117 09:06:17.993683 18462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:17.995960 18462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2: Bootstrap starting.
I20261117 09:06:17.996515 18462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:17.996729 18462 log.cc:826] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:17.997234 18462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2: No bootstrap required, opened a new log
I20261117 09:06:17.998378 18462 raft_consensus.cc:360] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516cfaada8ec4b4b9e6db46045a6cfb2" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38925 } }
I20261117 09:06:17.998495 18462 raft_consensus.cc:386] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:17.998561 18462 raft_consensus.cc:748] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 516cfaada8ec4b4b9e6db46045a6cfb2, State: Initialized, Role: FOLLOWER
I20261117 09:06:17.998672 18462 consensus_queue.cc:261] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516cfaada8ec4b4b9e6db46045a6cfb2" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38925 } }
I20261117 09:06:17.998732 18462 raft_consensus.cc:400] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261117 09:06:17.998754 18462 raft_consensus.cc:499] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261117 09:06:17.998793 18462 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:17.999269 18462 raft_consensus.cc:521] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516cfaada8ec4b4b9e6db46045a6cfb2" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38925 } }
I20261117 09:06:17.999363 18462 leader_election.cc:304] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 516cfaada8ec4b4b9e6db46045a6cfb2; no voters:
I20261117 09:06:17.999507 18462 leader_election.cc:290] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261117 09:06:17.999562 18465 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:17.999694 18465 raft_consensus.cc:705] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [term 1 LEADER]: Becoming Leader. State: Replica: 516cfaada8ec4b4b9e6db46045a6cfb2, State: Running, Role: LEADER
I20261117 09:06:17.999717 18462 sys_catalog.cc:565] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [sys.catalog]: configured and running, proceeding with master startup.
I20261117 09:06:17.999778 18465 consensus_queue.cc:238] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516cfaada8ec4b4b9e6db46045a6cfb2" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38925 } }
I20261117 09:06:18.000078 18466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "516cfaada8ec4b4b9e6db46045a6cfb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516cfaada8ec4b4b9e6db46045a6cfb2" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38925 } } }
I20261117 09:06:18.000093 18467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 516cfaada8ec4b4b9e6db46045a6cfb2. Latest consensus state: current_term: 1 leader_uuid: "516cfaada8ec4b4b9e6db46045a6cfb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516cfaada8ec4b4b9e6db46045a6cfb2" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38925 } } }
I20261117 09:06:18.000241 18467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:18.000232 18466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2 [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:18.000571 18473 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261117 09:06:18.000999 18473 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261117 09:06:18.002786 18473 catalog_manager.cc:1398] Generated new cluster ID: dde8504046424aa2814d91e4768c8e03
I20261117 09:06:18.002826 18473 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261117 09:06:18.003227 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 18395
I20261117 09:06:18.003326 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/master-0/wal/instance
I20261117 09:06:18.012135 18473 catalog_manager.cc:1421] Generated new certificate authority record
I20261117 09:06:18.012656 18473 catalog_manager.cc:1555] Loading token signing keys...
I20261117 09:06:18.017028 18473 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 516cfaada8ec4b4b9e6db46045a6cfb2: Generated new TSK 0
I20261117 09:06:18.017190 18473 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261117 09:06:18.022207 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.129:0
--local_ip_for_outbound_sockets=127.16.187.129
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38925
--builtin_ntp_servers=127.16.187.148:36577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261117 09:06:18.111824 18486 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261117 09:06:18.111975 18486 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:18.111989 18486 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:18.112004 18486 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:18.113551 18486 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:18.113608 18486 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.129
I20261117 09:06:18.115222 18486 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:36577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38925
--never_fsync=true
--heap_profile_path=/tmp/kudu.18486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:18.115391 18486 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:18.115547 18486 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:18.117911 18492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:18.117945 18491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:18.118005 18494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:18.118083 18486 server_base.cc:1061] running on GCE node
I20261117 09:06:18.118382 18486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:18.118628 18486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:18.119787 18486 hybrid_clock.cc:648] HybridClock initialized: now 1794906378119778 us; error 27 us; skew 500 ppm
I20261117 09:06:18.120980 18486 webserver.cc:577] Webserver started at http://127.16.187.129:45771/ using document root <none> and password file <none>
I20261117 09:06:18.121179 18486 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:18.121232 18486 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:18.121325 18486 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:18.122375 18486 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/data/instance:
uuid: "3183374fa14a439aa3d8fe0c505f92c5"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.122813 18486 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/wal/instance:
uuid: "3183374fa14a439aa3d8fe0c505f92c5"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.124212 18486 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:18.125046 18500 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.125222 18486 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:18.125296 18486 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/wal
uuid: "3183374fa14a439aa3d8fe0c505f92c5"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.125345 18486 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:18.139703 18486 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:18.140049 18486 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:18.140183 18486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:18.140407 18486 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:18.140905 18486 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:18.140972 18486 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.141006 18486 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:18.141023 18486 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.147637 18486 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.129:34225
I20261117 09:06:18.148002 18613 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.129:34225 every 8 connection(s)
I20261117 09:06:18.148252 18486 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/data/info.pb
I20261117 09:06:18.154312 18614 heartbeater.cc:345] Connected to a master server at 127.16.187.190:38925
I20261117 09:06:18.154498 18614 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:18.154767 18614 heartbeater.cc:508] Master 127.16.187.190:38925 requested a full tablet report, sending...
I20261117 09:06:18.155340 18426 ts_manager.cc:210] Registered new tserver with Master: 3183374fa14a439aa3d8fe0c505f92c5 (127.16.187.129:34225)
I20261117 09:06:18.155990 18426 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.129:51973
I20261117 09:06:18.157217 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 18486
I20261117 09:06:18.157296 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-0/wal/instance
I20261117 09:06:18.158471 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.130:0
--local_ip_for_outbound_sockets=127.16.187.130
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38925
--builtin_ntp_servers=127.16.187.148:36577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261117 09:06:18.247480 18617 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261117 09:06:18.247637 18617 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:18.247651 18617 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:18.247666 18617 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:18.249197 18617 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:18.249257 18617 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.130
I20261117 09:06:18.250898 18617 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:36577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38925
--never_fsync=true
--heap_profile_path=/tmp/kudu.18617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:18.251080 18617 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:18.251219 18617 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:18.253464 18623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:18.253473 18622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:18.253521 18625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:18.253712 18617 server_base.cc:1061] running on GCE node
I20261117 09:06:18.253847 18617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:18.254050 18617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:18.255211 18617 hybrid_clock.cc:648] HybridClock initialized: now 1794906378255194 us; error 26 us; skew 500 ppm
I20261117 09:06:18.256346 18617 webserver.cc:577] Webserver started at http://127.16.187.130:46383/ using document root <none> and password file <none>
I20261117 09:06:18.256549 18617 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:18.256620 18617 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:18.256771 18617 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:18.257629 18617 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/data/instance:
uuid: "39a043256dd54b56bcbca7d8ccacacb4"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.257966 18617 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/wal/instance:
uuid: "39a043256dd54b56bcbca7d8ccacacb4"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.259138 18617 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20261117 09:06:18.259828 18631 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.259981 18617 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:18.260049 18617 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/wal
uuid: "39a043256dd54b56bcbca7d8ccacacb4"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.260105 18617 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:18.279359 18617 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:18.279675 18617 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:18.279810 18617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:18.280141 18617 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:18.280509 18617 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:18.280546 18617 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.280577 18617 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:18.280602 18617 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.286221 18617 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.130:45745
I20261117 09:06:18.286279 18744 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.130:45745 every 8 connection(s)
I20261117 09:06:18.286631 18617 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/data/info.pb
I20261117 09:06:18.290843 18745 heartbeater.cc:345] Connected to a master server at 127.16.187.190:38925
I20261117 09:06:18.290930 18745 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:18.291085 18745 heartbeater.cc:508] Master 127.16.187.190:38925 requested a full tablet report, sending...
I20261117 09:06:18.291493 18426 ts_manager.cc:210] Registered new tserver with Master: 39a043256dd54b56bcbca7d8ccacacb4 (127.16.187.130:45745)
I20261117 09:06:18.291850 18426 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.130:44455
I20261117 09:06:18.293542 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 18617
I20261117 09:06:18.293622 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-1/wal/instance
I20261117 09:06:18.294849 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.131:0
--local_ip_for_outbound_sockets=127.16.187.131
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38925
--builtin_ntp_servers=127.16.187.148:36577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261117 09:06:18.371352 18748 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261117 09:06:18.371503 18748 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:18.371523 18748 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:18.371546 18748 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:18.373090 18748 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:18.373152 18748 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.131
I20261117 09:06:18.374814 18748 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:36577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38925
--never_fsync=true
--heap_profile_path=/tmp/kudu.18748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:18.375028 18748 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:18.375190 18748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:18.377460 18756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:18.377549 18754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:18.377627 18748 server_base.cc:1061] running on GCE node
W20261117 09:06:18.377499 18753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:18.377858 18748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:18.378010 18748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:18.379132 18748 hybrid_clock.cc:648] HybridClock initialized: now 1794906378379124 us; error 33 us; skew 500 ppm
I20261117 09:06:18.380049 18748 webserver.cc:577] Webserver started at http://127.16.187.131:44141/ using document root <none> and password file <none>
I20261117 09:06:18.380216 18748 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:18.380255 18748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:18.380322 18748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:18.381102 18748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/data/instance:
uuid: "926f958d122f4b729f47cc547a2ad4f3"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.381367 18748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/wal/instance:
uuid: "926f958d122f4b729f47cc547a2ad4f3"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.382448 18748 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:18.383234 18762 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.383402 18748 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:18.383466 18748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/wal
uuid: "926f958d122f4b729f47cc547a2ad4f3"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.383512 18748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:18.408245 18748 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:18.408501 18748 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:18.408596 18748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:18.408762 18748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:18.409106 18748 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:18.409139 18748 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.409163 18748 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:18.409185 18748 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.414742 18748 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.131:39283
I20261117 09:06:18.414798 18875 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.131:39283 every 8 connection(s)
I20261117 09:06:18.415092 18748 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/data/info.pb
I20261117 09:06:18.419162 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 18748
I20261117 09:06:18.419271 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-2/wal/instance
I20261117 09:06:18.419341 18876 heartbeater.cc:345] Connected to a master server at 127.16.187.190:38925
I20261117 09:06:18.419437 18876 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:18.419647 18876 heartbeater.cc:508] Master 127.16.187.190:38925 requested a full tablet report, sending...
I20261117 09:06:18.420087 18426 ts_manager.cc:210] Registered new tserver with Master: 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131:39283)
I20261117 09:06:18.420325 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.132:0
--local_ip_for_outbound_sockets=127.16.187.132
--webserver_interface=127.16.187.132
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38925
--builtin_ntp_servers=127.16.187.148:36577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
I20261117 09:06:18.420528 18426 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.131:42031
W20261117 09:06:18.493602 18879 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261117 09:06:18.493741 18879 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:18.493760 18879 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:18.493779 18879 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:18.495363 18879 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:18.495420 18879 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.132
I20261117 09:06:18.496982 18879 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:36577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.16.187.132
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38925
--never_fsync=true
--heap_profile_path=/tmp/kudu.18879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:18.497164 18879 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:18.497331 18879 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:18.499557 18885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:18.499565 18884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:18.499634 18879 server_base.cc:1061] running on GCE node
W20261117 09:06:18.499567 18887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:18.499811 18879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:18.500043 18879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:18.501178 18879 hybrid_clock.cc:648] HybridClock initialized: now 1794906378501161 us; error 39 us; skew 500 ppm
I20261117 09:06:18.502043 18879 webserver.cc:577] Webserver started at http://127.16.187.132:35521/ using document root <none> and password file <none>
I20261117 09:06:18.502218 18879 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:18.502255 18879 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:18.502341 18879 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:18.503239 18879 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/data/instance:
uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.503532 18879 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/wal/instance:
uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.504611 18879 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261117 09:06:18.505314 18893 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.505494 18879 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261117 09:06:18.505553 18879 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/wal
uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.505606 18879 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:18.529320 18879 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:18.529582 18879 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:18.529692 18879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:18.529882 18879 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:18.530194 18879 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:18.530231 18879 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.530263 18879 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:18.530287 18879 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.535575 18879 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.132:43639
I20261117 09:06:18.535625 19006 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.132:43639 every 8 connection(s)
I20261117 09:06:18.535905 18879 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/data/info.pb
I20261117 09:06:18.539968 19007 heartbeater.cc:345] Connected to a master server at 127.16.187.190:38925
I20261117 09:06:18.540071 19007 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:18.540244 19007 heartbeater.cc:508] Master 127.16.187.190:38925 requested a full tablet report, sending...
I20261117 09:06:18.540621 18426 ts_manager.cc:210] Registered new tserver with Master: ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132:43639)
I20261117 09:06:18.540984 18426 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.132:36527
I20261117 09:06:18.544606 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 18879
I20261117 09:06:18.544682 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794906366961029-17134-0/minicluster-data/ts-3/wal/instance
I20261117 09:06:18.545612 17134 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261117 09:06:18.555893 18426 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38178:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261117 09:06:18.562211 18810 tablet_service.cc:1500] Processing CreateTablet for tablet a6e24006bd4a4f5d9e7076e75d95a768 (DEFAULT_TABLE table=test-workload [id=cc7bb32ee2f748efb73c90f5eb24a8d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:18.562542 18810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6e24006bd4a4f5d9e7076e75d95a768. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:18.562579 18679 tablet_service.cc:1500] Processing CreateTablet for tablet a6e24006bd4a4f5d9e7076e75d95a768 (DEFAULT_TABLE table=test-workload [id=cc7bb32ee2f748efb73c90f5eb24a8d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:18.562700 18548 tablet_service.cc:1500] Processing CreateTablet for tablet a6e24006bd4a4f5d9e7076e75d95a768 (DEFAULT_TABLE table=test-workload [id=cc7bb32ee2f748efb73c90f5eb24a8d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:18.562860 18679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6e24006bd4a4f5d9e7076e75d95a768. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:18.562904 18548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6e24006bd4a4f5d9e7076e75d95a768. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:18.564656 19029 tablet_bootstrap.cc:492] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3: Bootstrap starting.
I20261117 09:06:18.565196 19029 tablet_bootstrap.cc:654] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:18.565272 19031 tablet_bootstrap.cc:492] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5: Bootstrap starting.
I20261117 09:06:18.565622 19029 log.cc:826] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:18.565840 19031 tablet_bootstrap.cc:654] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:18.566088 19031 log.cc:826] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:18.566226 19029 tablet_bootstrap.cc:492] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3: No bootstrap required, opened a new log
I20261117 09:06:18.566318 19029 ts_tablet_manager.cc:1449] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:18.566368 19030 tablet_bootstrap.cc:492] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4: Bootstrap starting.
I20261117 09:06:18.566962 19030 tablet_bootstrap.cc:654] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:18.567209 19030 log.cc:826] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:18.567631 19031 tablet_bootstrap.cc:492] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5: No bootstrap required, opened a new log
I20261117 09:06:18.567698 19031 ts_tablet_manager.cc:1449] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:18.567893 19029 raft_consensus.cc:360] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }
I20261117 09:06:18.568027 19029 raft_consensus.cc:748] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926f958d122f4b729f47cc547a2ad4f3, State: Initialized, Role: FOLLOWER
I20261117 09:06:18.568118 19029 consensus_queue.cc:261] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }
I20261117 09:06:18.568470 19029 ts_tablet_manager.cc:1480] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:18.568500 18876 heartbeater.cc:500] Master 127.16.187.190:38925 was elected leader, sending a full tablet report...
I20261117 09:06:18.569096 19031 raft_consensus.cc:360] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }
I20261117 09:06:18.569207 19031 raft_consensus.cc:748] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3183374fa14a439aa3d8fe0c505f92c5, State: Initialized, Role: FOLLOWER
I20261117 09:06:18.569262 19031 consensus_queue.cc:261] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }
I20261117 09:06:18.569459 19031 ts_tablet_manager.cc:1480] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:18.569495 18614 heartbeater.cc:500] Master 127.16.187.190:38925 was elected leader, sending a full tablet report...
I20261117 09:06:18.569929 19030 tablet_bootstrap.cc:492] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4: No bootstrap required, opened a new log
I20261117 09:06:18.570025 19030 ts_tablet_manager.cc:1449] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20261117 09:06:18.571256 19030 raft_consensus.cc:360] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }
I20261117 09:06:18.571360 19030 raft_consensus.cc:748] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39a043256dd54b56bcbca7d8ccacacb4, State: Initialized, Role: FOLLOWER
I20261117 09:06:18.571400 19030 consensus_queue.cc:261] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }
I20261117 09:06:18.571601 19030 ts_tablet_manager.cc:1480] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:18.571647 18745 heartbeater.cc:500] Master 127.16.187.190:38925 was elected leader, sending a full tablet report...
I20261117 09:06:18.582234 18699 tablet_service.cc:2033] Received Run Leader Election RPC: tablet_id: "a6e24006bd4a4f5d9e7076e75d95a768"
dest_uuid: "39a043256dd54b56bcbca7d8ccacacb4"
from {username='slave'} at 127.0.0.1:35826
I20261117 09:06:18.582338 18699 raft_consensus.cc:499] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20261117 09:06:18.582377 18699 raft_consensus.cc:3096] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:18.583069 18699 raft_consensus.cc:521] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }
I20261117 09:06:18.583341 18699 leader_election.cc:290] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [CANDIDATE]: Term 1 election: Requested vote from peers 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131:39283), 3183374fa14a439aa3d8fe0c505f92c5 (127.16.187.129:34225)
I20261117 09:06:18.586910 18568 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a6e24006bd4a4f5d9e7076e75d95a768" candidate_uuid: "39a043256dd54b56bcbca7d8ccacacb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3183374fa14a439aa3d8fe0c505f92c5"
I20261117 09:06:18.587064 18568 raft_consensus.cc:3096] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:18.587127 18830 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a6e24006bd4a4f5d9e7076e75d95a768" candidate_uuid: "39a043256dd54b56bcbca7d8ccacacb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "926f958d122f4b729f47cc547a2ad4f3"
I20261117 09:06:18.587247 18830 raft_consensus.cc:3096] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:18.587893 18568 raft_consensus.cc:2490] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39a043256dd54b56bcbca7d8ccacacb4 in term 1.
I20261117 09:06:18.588078 18830 raft_consensus.cc:2490] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39a043256dd54b56bcbca7d8ccacacb4 in term 1.
I20261117 09:06:18.588407 18633 leader_election.cc:304] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39a043256dd54b56bcbca7d8ccacacb4, 926f958d122f4b729f47cc547a2ad4f3; no voters:
I20261117 09:06:18.589725 19037 raft_consensus.cc:2826] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:18.589938 19037 raft_consensus.cc:705] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 1 LEADER]: Becoming Leader. State: Replica: 39a043256dd54b56bcbca7d8ccacacb4, State: Running, Role: LEADER
I20261117 09:06:18.590106 19037 consensus_queue.cc:238] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }
I20261117 09:06:18.591261 19044 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:18.591579 18424 catalog_manager.cc:5840] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39a043256dd54b56bcbca7d8ccacacb4 (127.16.187.130). New cstate: current_term: 1 leader_uuid: "39a043256dd54b56bcbca7d8ccacacb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } health_report { overall_health: HEALTHY } } }
I20261117 09:06:18.592532 19037 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:18.616259 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 18395
I20261117 09:06:18.621693 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.621758 17134 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } }
I20261117 09:06:18.623452 18699 consensus_queue.cc:238] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 1.21, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } }
I20261117 09:06:18.624111 18830 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 FOLLOWER]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 21. Preceding OpId from leader: term: 1 index: 22. (index mismatch)
I20261117 09:06:18.624300 18568 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 21. Preceding OpId from leader: term: 1 index: 22. (index mismatch)
I20261117 09:06:18.624336 19037 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20261117 09:06:18.624491 19044 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20261117 09:06:18.625193 19044 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 1 LEADER]: Committing config change with OpId 1.22: config changed from index -1 to 22, VOTER ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132) added. New config: { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } } }
I20261117 09:06:18.625407 18568 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Committing config change with OpId 1.22: config changed from index -1 to 22, VOTER ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132) added. New config: { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } } }
I20261117 09:06:18.626475 18830 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 FOLLOWER]: Committing config change with OpId 1.22: config changed from index -1 to 22, VOTER ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132) added. New config: { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } } }
I20261117 09:06:18.627192 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.627236 17134 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }
I20261117 09:06:18.627286 17134 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } }
W20261117 09:06:18.627575 18745 heartbeater.cc:648] Failed to heartbeat to 127.16.187.190:38925 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.187.190:38925: connect: Connection refused (error 111)
W20261117 09:06:18.627708 18614 heartbeater.cc:648] Failed to heartbeat to 127.16.187.190:38925 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.187.190:38925: connect: Connection refused (error 111)
W20261117 09:06:18.627766 18876 heartbeater.cc:648] Failed to heartbeat to 127.16.187.190:38925 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.187.190:38925: connect: Connection refused (error 111)
I20261117 09:06:18.628052 18699 consensus_queue.cc:238] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 1.22, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } }
I20261117 09:06:18.628530 18568 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 22. Preceding OpId from leader: term: 1 index: 23. (index mismatch)
I20261117 09:06:18.628567 18830 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 FOLLOWER]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 22. Preceding OpId from leader: term: 1 index: 23. (index mismatch)
W20261117 09:06:18.628646 18634 consensus_peers.cc:597] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 -> Peer ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132:43639): Couldn't send request to peer ab60c40a94ba4f0c9a45db1629e2dd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a6e24006bd4a4f5d9e7076e75d95a768. This is attempt 1: this message will repeat every 5th retry.
I20261117 09:06:18.628705 19044 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20261117 09:06:18.628793 19037 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20261117 09:06:18.629371 19046 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 1 LEADER]: Committing config change with OpId 1.23: config changed from index 22 to 23, 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131) changed from VOTER to NON_VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } } }
I20261117 09:06:18.629468 18568 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Committing config change with OpId 1.23: config changed from index 22 to 23, 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131) changed from VOTER to NON_VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } } }
I20261117 09:06:18.629493 18830 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 LEARNER]: Committing config change with OpId 1.23: config changed from index 22 to 23, 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131) changed from VOTER to NON_VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } } }
I20261117 09:06:18.631145 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.631186 17134 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: true } }
I20261117 09:06:18.631453 18699 consensus_queue.cc:238] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 1.23, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: true } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } }
I20261117 09:06:18.631829 18568 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 23. Preceding OpId from leader: term: 1 index: 24. (index mismatch)
I20261117 09:06:18.631851 18830 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 LEARNER]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 23. Preceding OpId from leader: term: 1 index: 24. (index mismatch)
W20261117 09:06:18.631939 18634 consensus_peers.cc:597] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 -> Peer ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132:43639): Couldn't send request to peer ab60c40a94ba4f0c9a45db1629e2dd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a6e24006bd4a4f5d9e7076e75d95a768. This is attempt 1: this message will repeat every 5th retry.
I20261117 09:06:18.632061 19037 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20261117 09:06:18.632233 19044 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20261117 09:06:18.632807 19037 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 1 LEADER]: Committing config change with OpId 1.24: config changed from index 23 to 24. New config: { opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: true } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } } }
I20261117 09:06:18.632952 18568 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Committing config change with OpId 1.24: config changed from index 23 to 24. New config: { opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: true } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } } }
I20261117 09:06:18.634167 18830 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 LEARNER]: Committing config change with OpId 1.24: config changed from index 23 to 24. New config: { opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: true } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } } }
I20261117 09:06:18.634284 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.634331 17134 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }
I20261117 09:06:18.634387 17134 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: NON_VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } }
I20261117 09:06:18.634862 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.634904 17134 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: NON_VOTER last_known_addr { host: "127.16.187.130" port: 45745 } attrs { promote: false replace: false } }
I20261117 09:06:18.635713 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.635754 17134 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: true replace: false } }
I20261117 09:06:18.636276 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.636310 17134 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }
I20261117 09:06:18.636344 17134 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } }
I20261117 09:06:18.636577 18699 consensus_queue.cc:238] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 1.24, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }
I20261117 09:06:18.636936 18568 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 24. Preceding OpId from leader: term: 1 index: 25. (index mismatch)
I20261117 09:06:18.637166 19037 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20261117 09:06:18.637883 19046 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 1 LEADER]: Committing config change with OpId 1.25: config changed from index 24 to 25, NON_VOTER 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131) evicted, VOTER ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132) evicted. New config: { opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } }
I20261117 09:06:18.638690 18568 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Committing config change with OpId 1.25: config changed from index 24 to 25, NON_VOTER 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131) evicted, VOTER ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132) evicted. New config: { opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } }
I20261117 09:06:18.638983 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.639026 17134 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }
I20261117 09:06:18.639068 17134 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } }
I20261117 09:06:18.639726 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.639761 17134 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }
I20261117 09:06:18.639986 18699 consensus_queue.cc:238] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 1.25, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }
I20261117 09:06:18.640357 18568 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 1 index: 26. (index mismatch)
I20261117 09:06:18.640522 18830 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 LEARNER]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 24. Preceding OpId from leader: term: 1 index: 26. (index mismatch)
I20261117 09:06:18.640563 19037 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20261117 09:06:18.640769 19037 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 24, Time since last communication: 0.000s
I20261117 09:06:18.641104 19046 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 1 LEADER]: Committing config change with OpId 1.26: config changed from index 25 to 26, VOTER 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131) added. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } }
I20261117 09:06:18.641199 18568 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Committing config change with OpId 1.26: config changed from index 25 to 26, VOTER 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131) added. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } }
I20261117 09:06:18.641316 18830 raft_consensus.cc:879] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: true } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } }.
Pending config: opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }:
New config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }
W20261117 09:06:18.641371 18830 raft_consensus.cc:1590] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 26' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: NON_VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: true } } peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: true replace: false } }.
Pending config: opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }:
New config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }
I20261117 09:06:18.641479 18830 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.25: config changed from index 24 to 25, NON_VOTER 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131) evicted, VOTER ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132) evicted. New config: { opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } }
I20261117 09:06:18.642284 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.642328 17134 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } }
I20261117 09:06:18.642607 18699 consensus_queue.cc:238] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } }
I20261117 09:06:18.643069 18568 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
W20261117 09:06:18.643110 18634 consensus_peers.cc:597] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 -> Peer ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132:43639): Couldn't send request to peer ab60c40a94ba4f0c9a45db1629e2dd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a6e24006bd4a4f5d9e7076e75d95a768. This is attempt 1: this message will repeat every 5th retry.
I20261117 09:06:18.643194 18830 raft_consensus.cc:1491] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20261117 09:06:18.643255 18830 raft_consensus.cc:1285] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 NON_PARTICIPANT]: Refusing update from remote peer 39a043256dd54b56bcbca7d8ccacacb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20261117 09:06:18.643222 19037 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20261117 09:06:18.643412 19066 consensus_queue.cc:1049] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 25, Time since last communication: 0.000s
I20261117 09:06:18.643646 18830 raft_consensus.cc:1491] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20261117 09:06:18.643810 18830 raft_consensus.cc:879] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 FOLLOWER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }.
Pending config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }:
New config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } }
W20261117 09:06:18.643839 18830 raft_consensus.cc:1590] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 27' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } }.
Pending config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }:
New config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } }
I20261117 09:06:18.643886 18830 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 FOLLOWER]: Committing config change with OpId 1.26: config changed from index 25 to 26, VOTER 926f958d122f4b729f47cc547a2ad4f3 (127.16.187.131) added. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } }
I20261117 09:06:18.644968 19066 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 39a043256dd54b56bcbca7d8ccacacb4 [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index 26 to 27, VOTER ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } } }
I20261117 09:06:18.645082 18830 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 926f958d122f4b729f47cc547a2ad4f3 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index 26 to 27, VOTER ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } } }
I20261117 09:06:18.646077 18568 raft_consensus.cc:2983] T a6e24006bd4a4f5d9e7076e75d95a768 P 3183374fa14a439aa3d8fe0c505f92c5 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index 26 to 27, VOTER ab60c40a94ba4f0c9a45db1629e2dd0f (127.16.187.132) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "3183374fa14a439aa3d8fe0c505f92c5" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 34225 } } peers { permanent_uuid: "39a043256dd54b56bcbca7d8ccacacb4" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 45745 } } peers { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } } peers { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } } }
I20261117 09:06:18.646416 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.646467 17134 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }
I20261117 09:06:18.646513 17134 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "ab60c40a94ba4f0c9a45db1629e2dd0f" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.16.187.132" port: 43639 } attrs { promote: false replace: false } }
I20261117 09:06:18.647186 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.647228 17134 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: false } }
I20261117 09:06:18.647696 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.648252 17134 raft_config_change-itest.cc:349] submitting config change with changes:
I20261117 09:06:18.648293 17134 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: false replace: true } }
I20261117 09:06:18.648320 17134 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "926f958d122f4b729f47cc547a2ad4f3" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 39283 } attrs { promote: true replace: false } }
I20261117 09:06:18.649169 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 18486
I20261117 09:06:18.654167 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 18617
I20261117 09:06:18.659353 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 18748
I20261117 09:06:18.663573 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 18879
2026-11-17T09:06:18Z chronyd exiting
[ OK ] RaftConfigChangeITest.TestBulkChangeConfig (799 ms)
[----------] 3 tests from RaftConfigChangeITest (11716 ms total)
[----------] 3 tests from SlowTabletBootstrapTest
[ RUN ] SlowTabletBootstrapTest.TestSlowBootstrap
I20261117 09:06:18.682828 17134 ts_itest-base.cc:115] Starting cluster with:
I20261117 09:06:18.682883 17134 ts_itest-base.cc:116] --------------
I20261117 09:06:18.682898 17134 ts_itest-base.cc:117] 4 tablet servers
I20261117 09:06:18.682911 17134 ts_itest-base.cc:118] 3 replicas per TS
I20261117 09:06:18.682921 17134 ts_itest-base.cc:119] --------------
2026-11-17T09:06:18Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-17T09:06:18Z Disabled control of system clock
I20261117 09:06:18.689921 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:36587
--webserver_interface=127.16.187.190
--webserver_port=0
--builtin_ntp_servers=127.16.187.148:41357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.187.190:36587 with env {}
W20261117 09:06:18.763702 19074 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:18.763835 19074 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:18.763851 19074 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:18.765331 19074 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261117 09:06:18.765375 19074 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:18.765388 19074 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261117 09:06:18.765398 19074 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261117 09:06:18.766959 19074 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:41357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.187.190:36587
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:36587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.16.187.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.19074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:18.767117 19074 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:18.767251 19074 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:18.769281 19079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:18.769291 19082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:18.769281 19080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:18.769563 19074 server_base.cc:1061] running on GCE node
I20261117 09:06:18.769729 19074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:18.769886 19074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:18.771001 19074 hybrid_clock.cc:648] HybridClock initialized: now 1794906378770989 us; error 28 us; skew 500 ppm
I20261117 09:06:18.771947 19074 webserver.cc:577] Webserver started at http://127.16.187.190:44039/ using document root <none> and password file <none>
I20261117 09:06:18.772115 19074 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:18.772152 19074 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:18.772235 19074 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:18.773039 19074 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "54d9c9a6cd144c6b9682e86843c3db0c"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.773334 19074 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "54d9c9a6cd144c6b9682e86843c3db0c"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.774392 19074 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:18.775067 19088 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.775226 19074 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.775285 19074 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
uuid: "54d9c9a6cd144c6b9682e86843c3db0c"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.775338 19074 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:18.799499 19074 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:18.799782 19074 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:18.799896 19074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:18.803630 19074 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.190:36587
I20261117 09:06:18.803678 19140 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.190:36587 every 8 connection(s)
I20261117 09:06:18.803975 19074 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20261117 09:06:18.804519 19141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:18.806674 19141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c: Bootstrap starting.
I20261117 09:06:18.807212 19141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:18.807415 19141 log.cc:826] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:18.807909 19141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c: No bootstrap required, opened a new log
I20261117 09:06:18.809047 19141 raft_consensus.cc:360] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d9c9a6cd144c6b9682e86843c3db0c" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36587 } }
I20261117 09:06:18.809160 19141 raft_consensus.cc:386] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:18.809178 19141 raft_consensus.cc:748] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d9c9a6cd144c6b9682e86843c3db0c, State: Initialized, Role: FOLLOWER
I20261117 09:06:18.809262 19141 consensus_queue.cc:261] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d9c9a6cd144c6b9682e86843c3db0c" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36587 } }
I20261117 09:06:18.809326 19141 raft_consensus.cc:400] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261117 09:06:18.809362 19141 raft_consensus.cc:499] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261117 09:06:18.809392 19141 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:18.809841 19141 raft_consensus.cc:521] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d9c9a6cd144c6b9682e86843c3db0c" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36587 } }
I20261117 09:06:18.809926 19141 leader_election.cc:304] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54d9c9a6cd144c6b9682e86843c3db0c; no voters:
I20261117 09:06:18.810045 19141 leader_election.cc:290] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20261117 09:06:18.810124 19144 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:18.810238 19141 sys_catalog.cc:565] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [sys.catalog]: configured and running, proceeding with master startup.
I20261117 09:06:18.810294 19144 raft_consensus.cc:705] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [term 1 LEADER]: Becoming Leader. State: Replica: 54d9c9a6cd144c6b9682e86843c3db0c, State: Running, Role: LEADER
I20261117 09:06:18.810422 19144 consensus_queue.cc:238] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d9c9a6cd144c6b9682e86843c3db0c" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36587 } }
I20261117 09:06:18.810798 19146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d9c9a6cd144c6b9682e86843c3db0c. Latest consensus state: current_term: 1 leader_uuid: "54d9c9a6cd144c6b9682e86843c3db0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d9c9a6cd144c6b9682e86843c3db0c" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36587 } } }
I20261117 09:06:18.810801 19145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54d9c9a6cd144c6b9682e86843c3db0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d9c9a6cd144c6b9682e86843c3db0c" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 36587 } } }
I20261117 09:06:18.810992 19145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:18.811266 19146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:18.811281 19153 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261117 09:06:18.811802 19153 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261117 09:06:18.813649 19153 catalog_manager.cc:1398] Generated new cluster ID: e68f4b3996ea40b395ad29e2d69627db
I20261117 09:06:18.813712 19153 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261117 09:06:18.813871 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 19074
I20261117 09:06:18.813983 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal/instance
I20261117 09:06:18.862761 19153 catalog_manager.cc:1421] Generated new certificate authority record
I20261117 09:06:18.863504 19153 catalog_manager.cc:1555] Loading token signing keys...
I20261117 09:06:18.869637 19153 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 54d9c9a6cd144c6b9682e86843c3db0c: Generated new TSK 0
I20261117 09:06:18.869856 19153 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261117 09:06:18.882323 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.129:0
--local_ip_for_outbound_sockets=127.16.187.129
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36587
--builtin_ntp_servers=127.16.187.148:41357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261117 09:06:18.960778 19165 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:18.960960 19165 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:18.960984 19165 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:18.961009 19165 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:18.962762 19165 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:18.962822 19165 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:18.962845 19165 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:18.962877 19165 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:18.962903 19165 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.129
I20261117 09:06:18.964722 19165 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:41357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36587
--never_fsync=true
--heap_profile_path=/tmp/kudu.19165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:18.965003 19165 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:18.965194 19165 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:18.967410 19173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:18.967404 19171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:18.967474 19170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:18.967731 19165 server_base.cc:1061] running on GCE node
I20261117 09:06:18.967919 19165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:18.968111 19165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:18.969336 19165 hybrid_clock.cc:648] HybridClock initialized: now 1794906378969301 us; error 33 us; skew 500 ppm
I20261117 09:06:18.970932 19165 webserver.cc:577] Webserver started at http://127.16.187.129:38091/ using document root <none> and password file <none>
I20261117 09:06:18.971155 19165 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:18.971199 19165 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:18.971283 19165 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:18.972124 19165 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "62cce14e351045e18e5a47f1d786e48f"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.972486 19165 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "62cce14e351045e18e5a47f1d786e48f"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.973639 19165 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20261117 09:06:18.974408 19179 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.974583 19165 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:18.974646 19165 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "62cce14e351045e18e5a47f1d786e48f"
format_stamp: "Formatted at 2026-11-17 09:06:18 on dist-test-slave-x52s"
I20261117 09:06:18.974707 19165 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:18.982620 19165 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:18.982949 19165 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:18.983074 19165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:18.983278 19165 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:18.983639 19165 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:18.983677 19165 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.983711 19165 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:18.983738 19165 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:18.990743 19165 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.129:37267
I20261117 09:06:18.990806 19292 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.129:37267 every 8 connection(s)
I20261117 09:06:18.991135 19165 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20261117 09:06:18.995936 19293 heartbeater.cc:345] Connected to a master server at 127.16.187.190:36587
I20261117 09:06:18.996042 19293 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:18.996199 19293 heartbeater.cc:508] Master 127.16.187.190:36587 requested a full tablet report, sending...
I20261117 09:06:18.996661 19105 ts_manager.cc:210] Registered new tserver with Master: 62cce14e351045e18e5a47f1d786e48f (127.16.187.129:37267)
I20261117 09:06:18.997016 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 19165
I20261117 09:06:18.997133 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20261117 09:06:18.997392 19105 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.129:35131
I20261117 09:06:18.998245 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.130:0
--local_ip_for_outbound_sockets=127.16.187.130
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36587
--builtin_ntp_servers=127.16.187.148:41357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261117 09:06:19.074180 19296 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:19.074331 19296 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:19.074344 19296 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:19.074359 19296 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:19.075901 19296 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:19.075958 19296 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:19.075974 19296 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:19.075994 19296 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:19.076005 19296 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.130
I20261117 09:06:19.077517 19296 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:41357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36587
--never_fsync=true
--heap_profile_path=/tmp/kudu.19296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:19.077724 19296 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:19.077907 19296 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:19.080102 19304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:19.080150 19302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:19.080235 19301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:19.080258 19296 server_base.cc:1061] running on GCE node
I20261117 09:06:19.080451 19296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:19.080623 19296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:19.081761 19296 hybrid_clock.cc:648] HybridClock initialized: now 1794906379081744 us; error 28 us; skew 500 ppm
I20261117 09:06:19.082947 19296 webserver.cc:577] Webserver started at http://127.16.187.130:44749/ using document root <none> and password file <none>
I20261117 09:06:19.083110 19296 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:19.083149 19296 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:19.083235 19296 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:19.084019 19296 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "46fbbe5020b04954943dfc7ddf2b7ef2"
format_stamp: "Formatted at 2026-11-17 09:06:19 on dist-test-slave-x52s"
I20261117 09:06:19.084311 19296 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "46fbbe5020b04954943dfc7ddf2b7ef2"
format_stamp: "Formatted at 2026-11-17 09:06:19 on dist-test-slave-x52s"
I20261117 09:06:19.085381 19296 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:19.086057 19310 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:19.086220 19296 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:19.086275 19296 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "46fbbe5020b04954943dfc7ddf2b7ef2"
format_stamp: "Formatted at 2026-11-17 09:06:19 on dist-test-slave-x52s"
I20261117 09:06:19.086328 19296 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:19.099431 19296 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:19.099661 19296 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:19.099764 19296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:19.099939 19296 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:19.100261 19296 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:19.100292 19296 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:19.100322 19296 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:19.100338 19296 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:19.105630 19296 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.130:43641
I20261117 09:06:19.105695 19423 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.130:43641 every 8 connection(s)
I20261117 09:06:19.105969 19296 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261117 09:06:19.110173 19424 heartbeater.cc:345] Connected to a master server at 127.16.187.190:36587
I20261117 09:06:19.110277 19424 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:19.110458 19424 heartbeater.cc:508] Master 127.16.187.190:36587 requested a full tablet report, sending...
I20261117 09:06:19.110847 19105 ts_manager.cc:210] Registered new tserver with Master: 46fbbe5020b04954943dfc7ddf2b7ef2 (127.16.187.130:43641)
I20261117 09:06:19.111260 19105 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.130:46707
I20261117 09:06:19.112618 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 19296
I20261117 09:06:19.112710 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20261117 09:06:19.113955 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.131:0
--local_ip_for_outbound_sockets=127.16.187.131
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36587
--builtin_ntp_servers=127.16.187.148:41357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261117 09:06:19.190300 19427 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:19.190455 19427 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:19.190469 19427 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:19.190483 19427 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:19.192109 19427 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:19.192157 19427 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:19.192173 19427 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:19.192193 19427 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:19.192205 19427 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.131
I20261117 09:06:19.193791 19427 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:41357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36587
--never_fsync=true
--heap_profile_path=/tmp/kudu.19427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:19.193974 19427 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:19.194144 19427 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:19.196512 19435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:19.196528 19427 server_base.cc:1061] running on GCE node
W20261117 09:06:19.196491 19433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:19.196499 19432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:19.196825 19427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:19.197009 19427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:19.198134 19427 hybrid_clock.cc:648] HybridClock initialized: now 1794906379198128 us; error 25 us; skew 500 ppm
I20261117 09:06:19.199172 19427 webserver.cc:577] Webserver started at http://127.16.187.131:33841/ using document root <none> and password file <none>
I20261117 09:06:19.199350 19427 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:19.199395 19427 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:19.199482 19427 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:19.200238 19427 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "153d38dd60b0433ba6e29921ed90ae70"
format_stamp: "Formatted at 2026-11-17 09:06:19 on dist-test-slave-x52s"
I20261117 09:06:19.200536 19427 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "153d38dd60b0433ba6e29921ed90ae70"
format_stamp: "Formatted at 2026-11-17 09:06:19 on dist-test-slave-x52s"
I20261117 09:06:19.201586 19427 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:19.202337 19441 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:19.202514 19427 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:19.202620 19427 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "153d38dd60b0433ba6e29921ed90ae70"
format_stamp: "Formatted at 2026-11-17 09:06:19 on dist-test-slave-x52s"
I20261117 09:06:19.202674 19427 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:19.216734 19427 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:19.216943 19427 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:19.217042 19427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:19.217221 19427 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:19.217532 19427 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:19.217562 19427 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:19.217590 19427 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:19.217617 19427 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:19.222791 19427 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.131:32927
I20261117 09:06:19.222838 19554 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.131:32927 every 8 connection(s)
I20261117 09:06:19.223129 19427 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261117 09:06:19.227293 19555 heartbeater.cc:345] Connected to a master server at 127.16.187.190:36587
I20261117 09:06:19.227397 19555 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:19.227560 19555 heartbeater.cc:508] Master 127.16.187.190:36587 requested a full tablet report, sending...
I20261117 09:06:19.227973 19105 ts_manager.cc:210] Registered new tserver with Master: 153d38dd60b0433ba6e29921ed90ae70 (127.16.187.131:32927)
I20261117 09:06:19.228307 19105 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.131:32805
I20261117 09:06:19.228502 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 19427
I20261117 09:06:19.228595 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20261117 09:06:19.229607 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.132:0
--local_ip_for_outbound_sockets=127.16.187.132
--webserver_interface=127.16.187.132
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36587
--builtin_ntp_servers=127.16.187.148:41357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261117 09:06:19.306087 19558 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:19.306258 19558 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:19.306278 19558 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:19.306303 19558 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:19.307924 19558 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:19.307968 19558 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:19.307991 19558 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:19.308018 19558 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:19.308037 19558 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.132
I20261117 09:06:19.309604 19558 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:41357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.16.187.132
--webserver_port=0
--tserver_master_addrs=127.16.187.190:36587
--never_fsync=true
--heap_profile_path=/tmp/kudu.19558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:19.309805 19558 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:19.309974 19558 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:19.312201 19566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:19.312280 19563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:19.312418 19564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:19.312471 19558 server_base.cc:1061] running on GCE node
I20261117 09:06:19.312625 19558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:19.312789 19558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:19.313922 19558 hybrid_clock.cc:648] HybridClock initialized: now 1794906379313911 us; error 25 us; skew 500 ppm
I20261117 09:06:19.314973 19558 webserver.cc:577] Webserver started at http://127.16.187.132:34039/ using document root <none> and password file <none>
I20261117 09:06:19.315135 19558 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:19.315171 19558 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:19.315251 19558 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:19.316039 19558 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "9c407bf9c6cc4b72ab8f494d4b3e0f20"
format_stamp: "Formatted at 2026-11-17 09:06:19 on dist-test-slave-x52s"
I20261117 09:06:19.316329 19558 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "9c407bf9c6cc4b72ab8f494d4b3e0f20"
format_stamp: "Formatted at 2026-11-17 09:06:19 on dist-test-slave-x52s"
I20261117 09:06:19.317334 19558 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:19.317958 19572 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:19.318092 19558 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:19.318147 19558 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "9c407bf9c6cc4b72ab8f494d4b3e0f20"
format_stamp: "Formatted at 2026-11-17 09:06:19 on dist-test-slave-x52s"
I20261117 09:06:19.318200 19558 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:19.329838 19558 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:19.330076 19558 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:19.330188 19558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:19.330370 19558 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:19.330696 19558 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:19.330727 19558 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:19.330758 19558 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:19.330776 19558 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:19.335939 19558 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.132:41359
I20261117 09:06:19.335994 19685 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.132:41359 every 8 connection(s)
I20261117 09:06:19.336284 19558 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261117 09:06:19.340390 19686 heartbeater.cc:345] Connected to a master server at 127.16.187.190:36587
I20261117 09:06:19.340490 19686 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:19.340677 19686 heartbeater.cc:508] Master 127.16.187.190:36587 requested a full tablet report, sending...
I20261117 09:06:19.341037 19105 ts_manager.cc:210] Registered new tserver with Master: 9c407bf9c6cc4b72ab8f494d4b3e0f20 (127.16.187.132:41359)
I20261117 09:06:19.341367 19105 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.132:57061
I20261117 09:06:19.343802 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 19558
I20261117 09:06:19.343883 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20261117 09:06:19.345240 17134 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261117 09:06:19.350670 19105 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59754:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20261117 09:06:19.358022 19227 tablet_service.cc:1500] Processing CreateTablet for tablet e8bb66de730341079791c6f7cbe69d11 (DEFAULT_TABLE table=TestTable [id=5487ab578a88455db9f6b6b58cd4aeb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:19.358100 19358 tablet_service.cc:1500] Processing CreateTablet for tablet e8bb66de730341079791c6f7cbe69d11 (DEFAULT_TABLE table=TestTable [id=5487ab578a88455db9f6b6b58cd4aeb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:19.358203 19489 tablet_service.cc:1500] Processing CreateTablet for tablet e8bb66de730341079791c6f7cbe69d11 (DEFAULT_TABLE table=TestTable [id=5487ab578a88455db9f6b6b58cd4aeb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:19.358323 19227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bb66de730341079791c6f7cbe69d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:19.358351 19358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bb66de730341079791c6f7cbe69d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:19.358404 19489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bb66de730341079791c6f7cbe69d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:19.360380 19700 tablet_bootstrap.cc:492] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f: Bootstrap starting.
I20261117 09:06:19.360810 19701 tablet_bootstrap.cc:492] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2: Bootstrap starting.
I20261117 09:06:19.360972 19700 tablet_bootstrap.cc:654] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:19.361260 19700 log.cc:826] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:19.361404 19701 tablet_bootstrap.cc:654] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:19.361647 19701 log.cc:826] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:19.361891 19700 tablet_bootstrap.cc:492] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f: No bootstrap required, opened a new log
I20261117 09:06:19.361927 19702 tablet_bootstrap.cc:492] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Bootstrap starting.
I20261117 09:06:19.361958 19700 ts_tablet_manager.cc:1449] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:19.362562 19702 tablet_bootstrap.cc:654] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:19.362809 19701 tablet_bootstrap.cc:492] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2: No bootstrap required, opened a new log
I20261117 09:06:19.362841 19702 log.cc:826] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:19.362869 19701 ts_tablet_manager.cc:1449] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:19.363416 19702 tablet_bootstrap.cc:492] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: No bootstrap required, opened a new log
I20261117 09:06:19.363483 19702 ts_tablet_manager.cc:1449] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20261117 09:06:19.363536 19700 raft_consensus.cc:360] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:19.363672 19700 raft_consensus.cc:386] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:19.363698 19700 raft_consensus.cc:748] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cce14e351045e18e5a47f1d786e48f, State: Initialized, Role: FOLLOWER
I20261117 09:06:19.363780 19700 consensus_queue.cc:261] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:19.363986 19700 ts_tablet_manager.cc:1480] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:19.364130 19293 heartbeater.cc:500] Master 127.16.187.190:36587 was elected leader, sending a full tablet report...
I20261117 09:06:19.364378 19701 raft_consensus.cc:360] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:19.364492 19701 raft_consensus.cc:386] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:19.364522 19701 raft_consensus.cc:748] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46fbbe5020b04954943dfc7ddf2b7ef2, State: Initialized, Role: FOLLOWER
I20261117 09:06:19.364610 19701 consensus_queue.cc:261] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:19.364830 19424 heartbeater.cc:500] Master 127.16.187.190:36587 was elected leader, sending a full tablet report...
I20261117 09:06:19.364899 19702 raft_consensus.cc:360] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:19.365021 19701 ts_tablet_manager.cc:1480] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:19.365017 19702 raft_consensus.cc:386] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:19.365056 19702 raft_consensus.cc:748] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153d38dd60b0433ba6e29921ed90ae70, State: Initialized, Role: FOLLOWER
I20261117 09:06:19.365163 19702 consensus_queue.cc:261] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:19.365398 19702 ts_tablet_manager.cc:1480] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261117 09:06:19.365494 19555 heartbeater.cc:500] Master 127.16.187.190:36587 was elected leader, sending a full tablet report...
W20261117 09:06:19.392192 19294 tablet.cc:2441] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261117 09:06:19.406777 19425 tablet.cc:2441] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261117 09:06:19.423733 19556 tablet.cc:2441] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261117 09:06:19.470301 19706 raft_consensus.cc:499] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261117 09:06:19.470435 19706 raft_consensus.cc:521] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:19.470810 19706 leader_election.cc:290] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 153d38dd60b0433ba6e29921ed90ae70 (127.16.187.131:32927), 46fbbe5020b04954943dfc7ddf2b7ef2 (127.16.187.130:43641)
I20261117 09:06:19.473661 19378 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e8bb66de730341079791c6f7cbe69d11" candidate_uuid: "62cce14e351045e18e5a47f1d786e48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" is_pre_election: true
I20261117 09:06:19.473806 19378 raft_consensus.cc:2490] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cce14e351045e18e5a47f1d786e48f in term 0.
I20261117 09:06:19.473898 19509 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e8bb66de730341079791c6f7cbe69d11" candidate_uuid: "62cce14e351045e18e5a47f1d786e48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153d38dd60b0433ba6e29921ed90ae70" is_pre_election: true
I20261117 09:06:19.474038 19509 raft_consensus.cc:2490] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cce14e351045e18e5a47f1d786e48f in term 0.
I20261117 09:06:19.474059 19181 leader_election.cc:304] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46fbbe5020b04954943dfc7ddf2b7ef2, 62cce14e351045e18e5a47f1d786e48f; no voters:
I20261117 09:06:19.474161 19706 raft_consensus.cc:2826] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261117 09:06:19.474216 19706 raft_consensus.cc:499] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261117 09:06:19.474241 19706 raft_consensus.cc:3096] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:19.474892 19706 raft_consensus.cc:521] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:19.475019 19706 leader_election.cc:290] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [CANDIDATE]: Term 1 election: Requested vote from peers 153d38dd60b0433ba6e29921ed90ae70 (127.16.187.131:32927), 46fbbe5020b04954943dfc7ddf2b7ef2 (127.16.187.130:43641)
I20261117 09:06:19.475160 19378 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e8bb66de730341079791c6f7cbe69d11" candidate_uuid: "62cce14e351045e18e5a47f1d786e48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2"
I20261117 09:06:19.475191 19509 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e8bb66de730341079791c6f7cbe69d11" candidate_uuid: "62cce14e351045e18e5a47f1d786e48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153d38dd60b0433ba6e29921ed90ae70"
I20261117 09:06:19.475229 19378 raft_consensus.cc:3096] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:19.475262 19509 raft_consensus.cc:3096] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:19.476115 19378 raft_consensus.cc:2490] T e8bb66de730341079791c6f7cbe69d11 P 46fbbe5020b04954943dfc7ddf2b7ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cce14e351045e18e5a47f1d786e48f in term 1.
I20261117 09:06:19.476172 19509 raft_consensus.cc:2490] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cce14e351045e18e5a47f1d786e48f in term 1.
I20261117 09:06:19.476339 19182 leader_election.cc:304] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 153d38dd60b0433ba6e29921ed90ae70, 62cce14e351045e18e5a47f1d786e48f; no voters:
I20261117 09:06:19.476450 19706 raft_consensus.cc:2826] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:19.476569 19706 raft_consensus.cc:705] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [term 1 LEADER]: Becoming Leader. State: Replica: 62cce14e351045e18e5a47f1d786e48f, State: Running, Role: LEADER
I20261117 09:06:19.476652 19706 consensus_queue.cc:238] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:19.477415 19105 catalog_manager.cc:5840] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f reported cstate change: term changed from 0 to 1, leader changed from <none> to 62cce14e351045e18e5a47f1d786e48f (127.16.187.129). New cstate: current_term: 1 leader_uuid: "62cce14e351045e18e5a47f1d786e48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } health_report { overall_health: UNKNOWN } } }
I20261117 09:06:19.477754 19712 consensus_queue.cc:1049] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:19.477913 19710 consensus_queue.cc:1049] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:19.503026 17134 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261117 09:06:19.503533 17134 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 62cce14e351045e18e5a47f1d786e48f to finish bootstrapping
I20261117 09:06:19.507180 17134 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 46fbbe5020b04954943dfc7ddf2b7ef2 to finish bootstrapping
I20261117 09:06:19.510385 17134 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 153d38dd60b0433ba6e29921ed90ae70 to finish bootstrapping
I20261117 09:06:19.513563 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 19427
I20261117 09:06:19.518321 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.131:32927
--local_ip_for_outbound_sockets=127.16.187.131
--tserver_master_addrs=127.16.187.190:36587
--webserver_port=33841
--webserver_interface=127.16.187.131
--builtin_ntp_servers=127.16.187.148:41357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000
--tablet_bootstrap_inject_latency_ms=4000 with env {}
W20261117 09:06:19.593688 19719 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:19.593845 19719 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:19.593865 19719 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:19.593897 19719 flags.cc:432] Enabled unsafe flag: --tablet_bootstrap_inject_latency_ms=4000
W20261117 09:06:19.593921 19719 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:19.595468 19719 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:19.595511 19719 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:19.595533 19719 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:19.595562 19719 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:19.595587 19719 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.131
I20261117 09:06:19.597139 19719 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:41357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.131:32927
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.16.187.131
--webserver_port=33841
--tserver_master_addrs=127.16.187.190:36587
--tablet_bootstrap_inject_latency_ms=4000
--never_fsync=true
--heap_profile_path=/tmp/kudu.19719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:19.597330 19719 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:19.597486 19719 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:19.599578 19725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:19.599608 19724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:19.599651 19727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:19.599942 19719 server_base.cc:1061] running on GCE node
I20261117 09:06:19.600112 19719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:19.600270 19719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:19.601378 19719 hybrid_clock.cc:648] HybridClock initialized: now 1794906379601371 us; error 24 us; skew 500 ppm
I20261117 09:06:19.602495 19719 webserver.cc:577] Webserver started at http://127.16.187.131:33841/ using document root <none> and password file <none>
I20261117 09:06:19.602716 19719 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:19.602768 19719 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:19.603948 19719 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:19.604691 19733 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:19.604897 19719 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:19.604965 19719 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "153d38dd60b0433ba6e29921ed90ae70"
format_stamp: "Formatted at 2026-11-17 09:06:19 on dist-test-slave-x52s"
I20261117 09:06:19.605232 19719 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:19.620719 19719 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:19.620968 19719 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:19.621094 19719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:19.621315 19719 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:19.621724 19740 ts_tablet_manager.cc:588] Loading tablet metadata (0/1 complete)
I20261117 09:06:19.622481 19719 ts_tablet_manager.cc:631] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20261117 09:06:19.622551 19719 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20261117 09:06:19.622607 19719 ts_tablet_manager.cc:646] Registering tablets (0/1 complete)
I20261117 09:06:19.623159 19719 ts_tablet_manager.cc:662] Registered 1 tablets
W20261117 09:06:19.623181 19740 ts_tablet_manager.cc:1428] Injecting 4000ms delay in tablet bootstrapping
I20261117 09:06:19.623190 19719 ts_tablet_manager.cc:641] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20261117 09:06:19.628587 19719 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.131:32927
I20261117 09:06:19.628626 19847 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.131:32927 every 8 connection(s)
I20261117 09:06:19.628935 19719 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261117 09:06:19.632601 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 19719
I20261117 09:06:19.634733 19848 heartbeater.cc:345] Connected to a master server at 127.16.187.190:36587
I20261117 09:06:19.634819 19848 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:19.634999 19848 heartbeater.cc:508] Master 127.16.187.190:36587 requested a full tablet report, sending...
I20261117 09:06:19.635457 19104 ts_manager.cc:210] Re-registered known tserver with Master: 153d38dd60b0433ba6e29921ed90ae70 (127.16.187.131:32927)
I20261117 09:06:19.635848 19104 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.131:44377
W20261117 09:06:19.894029 19182 consensus_peers.cc:597] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f -> Peer 153d38dd60b0433ba6e29921ed90ae70 (127.16.187.131:32927): Couldn't send request to peer 153d38dd60b0433ba6e29921ed90ae70. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20261117 09:06:20.342100 19686 heartbeater.cc:500] Master 127.16.187.190:36587 was elected leader, sending a full tablet report...
I20261117 09:06:20.636587 19848 heartbeater.cc:500] Master 127.16.187.190:36587 was elected leader, sending a full tablet report...
W20261117 09:06:22.332532 19182 consensus_peers.cc:597] T e8bb66de730341079791c6f7cbe69d11 P 62cce14e351045e18e5a47f1d786e48f -> Peer 153d38dd60b0433ba6e29921ed90ae70 (127.16.187.131:32927): Couldn't send request to peer 153d38dd60b0433ba6e29921ed90ae70. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20261117 09:06:23.623433 19740 tablet_bootstrap.cc:492] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Bootstrap starting.
I20261117 09:06:23.626187 19740 log.cc:826] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:23.627089 19740 tablet_bootstrap.cc:492] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261117 09:06:23.627290 19740 tablet_bootstrap.cc:492] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Bootstrap complete.
I20261117 09:06:23.627454 19740 ts_tablet_manager.cc:1449] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20261117 09:06:23.628407 19740 raft_consensus.cc:360] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:23.628589 19740 raft_consensus.cc:748] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153d38dd60b0433ba6e29921ed90ae70, State: Initialized, Role: FOLLOWER
I20261117 09:06:23.628674 19740 consensus_queue.cc:261] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153d38dd60b0433ba6e29921ed90ae70" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 32927 } } peers { permanent_uuid: "62cce14e351045e18e5a47f1d786e48f" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 37267 } } peers { permanent_uuid: "46fbbe5020b04954943dfc7ddf2b7ef2" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 43641 } }
I20261117 09:06:23.628859 19740 ts_tablet_manager.cc:1480] T e8bb66de730341079791c6f7cbe69d11 P 153d38dd60b0433ba6e29921ed90ae70: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:26.638106 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 19165
I20261117 09:06:26.643365 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 19296
I20261117 09:06:26.647506 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 19719
I20261117 09:06:26.651981 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 19558
I20261117 09:06:26.656226 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 19074
2026-11-17T09:06:26Z chronyd exiting
[ OK ] SlowTabletBootstrapTest.TestSlowBootstrap (7993 ms)
[ RUN ] SlowTabletBootstrapTest.TestFallBehind
I20261117 09:06:26.675956 17134 ts_itest-base.cc:115] Starting cluster with:
I20261117 09:06:26.676002 17134 ts_itest-base.cc:116] --------------
I20261117 09:06:26.676017 17134 ts_itest-base.cc:117] 4 tablet servers
I20261117 09:06:26.676030 17134 ts_itest-base.cc:118] 3 replicas per TS
I20261117 09:06:26.676043 17134 ts_itest-base.cc:119] --------------
2026-11-17T09:06:26Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-17T09:06:26Z Disabled control of system clock
I20261117 09:06:26.682737 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:40495
--webserver_interface=127.16.187.190
--webserver_port=0
--builtin_ntp_servers=127.16.187.148:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.187.190:40495 with env {}
W20261117 09:06:26.757306 19865 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:26.757476 19865 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:26.757510 19865 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:26.759127 19865 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261117 09:06:26.759182 19865 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:26.759199 19865 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261117 09:06:26.759213 19865 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261117 09:06:26.760923 19865 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.187.190:40495
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:40495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.16.187.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.19865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:26.761155 19865 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:26.761328 19865 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20261117 09:06:26.764190 19865 server_base.cc:1061] running on GCE node
W20261117 09:06:26.764163 19870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:26.764163 19873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:26.764163 19871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:26.764570 19865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:26.764797 19865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:26.765934 19865 hybrid_clock.cc:648] HybridClock initialized: now 1794906386765918 us; error 30 us; skew 500 ppm
I20261117 09:06:26.766964 19865 webserver.cc:577] Webserver started at http://127.16.187.190:33075/ using document root <none> and password file <none>
I20261117 09:06:26.767138 19865 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:26.767177 19865 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:26.767277 19865 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:26.768095 19865 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "1dd73c1af4a040f2a26ff84d646161a8"
format_stamp: "Formatted at 2026-11-17 09:06:26 on dist-test-slave-x52s"
I20261117 09:06:26.768416 19865 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "1dd73c1af4a040f2a26ff84d646161a8"
format_stamp: "Formatted at 2026-11-17 09:06:26 on dist-test-slave-x52s"
I20261117 09:06:26.769505 19865 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:26.770176 19879 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:26.770418 19865 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20261117 09:06:26.770490 19865 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
uuid: "1dd73c1af4a040f2a26ff84d646161a8"
format_stamp: "Formatted at 2026-11-17 09:06:26 on dist-test-slave-x52s"
I20261117 09:06:26.770601 19865 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:26.782058 19865 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:26.782326 19865 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:26.782436 19865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:26.786214 19865 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.190:40495
I20261117 09:06:26.786252 19931 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.190:40495 every 8 connection(s)
I20261117 09:06:26.786594 19865 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20261117 09:06:26.787151 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:26.789168 19932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8: Bootstrap starting.
I20261117 09:06:26.789712 19932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:26.789908 19932 log.cc:826] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:26.790454 19932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8: No bootstrap required, opened a new log
I20261117 09:06:26.791636 19932 raft_consensus.cc:360] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd73c1af4a040f2a26ff84d646161a8" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 40495 } }
I20261117 09:06:26.791752 19932 raft_consensus.cc:386] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:26.791769 19932 raft_consensus.cc:748] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd73c1af4a040f2a26ff84d646161a8, State: Initialized, Role: FOLLOWER
I20261117 09:06:26.791852 19932 consensus_queue.cc:261] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd73c1af4a040f2a26ff84d646161a8" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 40495 } }
I20261117 09:06:26.791957 19932 raft_consensus.cc:400] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261117 09:06:26.792009 19932 raft_consensus.cc:499] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261117 09:06:26.792089 19932 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:26.792590 19932 raft_consensus.cc:521] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd73c1af4a040f2a26ff84d646161a8" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 40495 } }
I20261117 09:06:26.792685 19932 leader_election.cc:304] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dd73c1af4a040f2a26ff84d646161a8; no voters:
I20261117 09:06:26.792804 19932 leader_election.cc:290] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261117 09:06:26.792897 19935 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:26.793008 19932 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [sys.catalog]: configured and running, proceeding with master startup.
I20261117 09:06:26.793072 19935 raft_consensus.cc:705] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [term 1 LEADER]: Becoming Leader. State: Replica: 1dd73c1af4a040f2a26ff84d646161a8, State: Running, Role: LEADER
I20261117 09:06:26.793195 19935 consensus_queue.cc:238] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd73c1af4a040f2a26ff84d646161a8" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 40495 } }
I20261117 09:06:26.793576 19937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dd73c1af4a040f2a26ff84d646161a8. Latest consensus state: current_term: 1 leader_uuid: "1dd73c1af4a040f2a26ff84d646161a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd73c1af4a040f2a26ff84d646161a8" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 40495 } } }
I20261117 09:06:26.793588 19936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dd73c1af4a040f2a26ff84d646161a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd73c1af4a040f2a26ff84d646161a8" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 40495 } } }
I20261117 09:06:26.793685 19937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:26.793694 19936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8 [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:26.793871 19944 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261117 09:06:26.794377 19944 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261117 09:06:26.796059 19944 catalog_manager.cc:1398] Generated new cluster ID: f33e6e1d8dfe4ca2a7bf5a4afdc4492d
I20261117 09:06:26.796116 19944 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261117 09:06:26.796372 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 19865
I20261117 09:06:26.796458 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal/instance
I20261117 09:06:26.801366 19944 catalog_manager.cc:1421] Generated new certificate authority record
I20261117 09:06:26.802088 19944 catalog_manager.cc:1555] Loading token signing keys...
I20261117 09:06:26.814208 19944 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 1dd73c1af4a040f2a26ff84d646161a8: Generated new TSK 0
I20261117 09:06:26.814386 19944 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261117 09:06:26.818569 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.129:0
--local_ip_for_outbound_sockets=127.16.187.129
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:40495
--builtin_ntp_servers=127.16.187.148:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261117 09:06:26.901959 19956 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:26.902132 19956 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:26.902153 19956 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:26.902174 19956 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:26.903849 19956 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:26.903898 19956 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:26.903920 19956 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:26.903949 19956 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:26.903967 19956 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.129
I20261117 09:06:26.905613 19956 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:40495
--never_fsync=true
--heap_profile_path=/tmp/kudu.19956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:26.905812 19956 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:26.905982 19956 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:26.908260 19962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:26.908314 19956 server_base.cc:1061] running on GCE node
W20261117 09:06:26.908244 19961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:26.908543 19964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:26.908703 19956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:26.908893 19956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:26.910040 19956 hybrid_clock.cc:648] HybridClock initialized: now 1794906386910023 us; error 32 us; skew 500 ppm
I20261117 09:06:26.911197 19956 webserver.cc:577] Webserver started at http://127.16.187.129:45553/ using document root <none> and password file <none>
I20261117 09:06:26.911376 19956 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:26.911412 19956 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:26.911486 19956 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:26.912300 19956 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "1290a67ac40240b0842588e83fc93605"
format_stamp: "Formatted at 2026-11-17 09:06:26 on dist-test-slave-x52s"
I20261117 09:06:26.912590 19956 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "1290a67ac40240b0842588e83fc93605"
format_stamp: "Formatted at 2026-11-17 09:06:26 on dist-test-slave-x52s"
I20261117 09:06:26.913712 19956 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261117 09:06:26.914567 19970 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:26.914729 19956 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20261117 09:06:26.914788 19956 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "1290a67ac40240b0842588e83fc93605"
format_stamp: "Formatted at 2026-11-17 09:06:26 on dist-test-slave-x52s"
I20261117 09:06:26.914844 19956 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:26.926507 19956 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:26.926746 19956 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:26.926841 19956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:26.927012 19956 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:26.927320 19956 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:26.927349 19956 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:26.927378 19956 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:26.927395 19956 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:26.932399 19956 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.129:34971
I20261117 09:06:26.932533 20083 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.129:34971 every 8 connection(s)
I20261117 09:06:26.932744 19956 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20261117 09:06:26.933424 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 19956
I20261117 09:06:26.933521 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20261117 09:06:26.934753 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.130:0
--local_ip_for_outbound_sockets=127.16.187.130
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:40495
--builtin_ntp_servers=127.16.187.148:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261117 09:06:26.937877 20084 heartbeater.cc:345] Connected to a master server at 127.16.187.190:40495
I20261117 09:06:26.937975 20084 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:26.938179 20084 heartbeater.cc:508] Master 127.16.187.190:40495 requested a full tablet report, sending...
I20261117 09:06:26.938688 19896 ts_manager.cc:210] Registered new tserver with Master: 1290a67ac40240b0842588e83fc93605 (127.16.187.129:34971)
I20261117 09:06:26.939654 19896 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.129:34323
W20261117 09:06:27.010490 20087 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:27.010650 20087 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:27.010663 20087 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:27.010679 20087 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:27.012219 20087 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:27.012264 20087 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:27.012279 20087 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:27.012298 20087 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:27.012310 20087 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.130
I20261117 09:06:27.013826 20087 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:40495
--never_fsync=true
--heap_profile_path=/tmp/kudu.20087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:27.013988 20087 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:27.014122 20087 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:27.016481 20095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:27.016533 20093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:27.016605 20087 server_base.cc:1061] running on GCE node
W20261117 09:06:27.016698 20092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:27.016983 20087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:27.017187 20087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:27.018311 20087 hybrid_clock.cc:648] HybridClock initialized: now 1794906387018302 us; error 36 us; skew 500 ppm
I20261117 09:06:27.019393 20087 webserver.cc:577] Webserver started at http://127.16.187.130:45037/ using document root <none> and password file <none>
I20261117 09:06:27.019575 20087 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:27.019614 20087 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:27.019699 20087 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:27.020516 20087 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "0fd3880772e74fc0aff725468109a033"
format_stamp: "Formatted at 2026-11-17 09:06:27 on dist-test-slave-x52s"
I20261117 09:06:27.020816 20087 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "0fd3880772e74fc0aff725468109a033"
format_stamp: "Formatted at 2026-11-17 09:06:27 on dist-test-slave-x52s"
I20261117 09:06:27.021883 20087 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261117 09:06:27.022655 20101 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:27.022809 20087 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261117 09:06:27.022866 20087 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0fd3880772e74fc0aff725468109a033"
format_stamp: "Formatted at 2026-11-17 09:06:27 on dist-test-slave-x52s"
I20261117 09:06:27.022926 20087 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:27.039217 20087 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:27.039429 20087 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:27.039528 20087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:27.039709 20087 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:27.040011 20087 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:27.040051 20087 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:27.040081 20087 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:27.040098 20087 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:27.045339 20087 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.130:39781
I20261117 09:06:27.045397 20214 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.130:39781 every 8 connection(s)
I20261117 09:06:27.045674 20087 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261117 09:06:27.049276 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 20087
I20261117 09:06:27.049371 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20261117 09:06:27.049853 20215 heartbeater.cc:345] Connected to a master server at 127.16.187.190:40495
I20261117 09:06:27.049966 20215 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:27.050154 20215 heartbeater.cc:508] Master 127.16.187.190:40495 requested a full tablet report, sending...
I20261117 09:06:27.050578 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.131:0
--local_ip_for_outbound_sockets=127.16.187.131
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:40495
--builtin_ntp_servers=127.16.187.148:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261117 09:06:27.050606 19896 ts_manager.cc:210] Registered new tserver with Master: 0fd3880772e74fc0aff725468109a033 (127.16.187.130:39781)
I20261117 09:06:27.051048 19896 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.130:49155
W20261117 09:06:27.125396 20218 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:27.125552 20218 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:27.125567 20218 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:27.125589 20218 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:27.127267 20218 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:27.127315 20218 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:27.127331 20218 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:27.127350 20218 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:27.127362 20218 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.131
I20261117 09:06:27.128965 20218 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:40495
--never_fsync=true
--heap_profile_path=/tmp/kudu.20218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:27.129154 20218 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:27.129338 20218 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:27.131597 20223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:27.131804 20224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:27.131853 20226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:27.132352 20218 server_base.cc:1061] running on GCE node
I20261117 09:06:27.132498 20218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:27.132661 20218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:27.133806 20218 hybrid_clock.cc:648] HybridClock initialized: now 1794906387133793 us; error 28 us; skew 500 ppm
I20261117 09:06:27.134864 20218 webserver.cc:577] Webserver started at http://127.16.187.131:38871/ using document root <none> and password file <none>
I20261117 09:06:27.135020 20218 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:27.135053 20218 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:27.135114 20218 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:27.135886 20218 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "5cb229095048484c972cfa2de0e574a9"
format_stamp: "Formatted at 2026-11-17 09:06:27 on dist-test-slave-x52s"
I20261117 09:06:27.137575 20218 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "5cb229095048484c972cfa2de0e574a9"
format_stamp: "Formatted at 2026-11-17 09:06:27 on dist-test-slave-x52s"
I20261117 09:06:27.138705 20218 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:27.139479 20232 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:27.139671 20218 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:27.139735 20218 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "5cb229095048484c972cfa2de0e574a9"
format_stamp: "Formatted at 2026-11-17 09:06:27 on dist-test-slave-x52s"
I20261117 09:06:27.139780 20218 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:27.155395 20218 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:27.155582 20218 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:27.155661 20218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:27.155810 20218 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:27.156085 20218 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:27.156114 20218 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:27.156143 20218 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:27.156167 20218 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:27.161288 20218 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.131:36919
I20261117 09:06:27.161357 20345 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.131:36919 every 8 connection(s)
I20261117 09:06:27.161626 20218 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261117 09:06:27.164857 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 20218
I20261117 09:06:27.165009 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20261117 09:06:27.165860 20346 heartbeater.cc:345] Connected to a master server at 127.16.187.190:40495
I20261117 09:06:27.165946 20346 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:27.166112 20346 heartbeater.cc:508] Master 127.16.187.190:40495 requested a full tablet report, sending...
I20261117 09:06:27.166115 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.132:0
--local_ip_for_outbound_sockets=127.16.187.132
--webserver_interface=127.16.187.132
--webserver_port=0
--tserver_master_addrs=127.16.187.190:40495
--builtin_ntp_servers=127.16.187.148:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261117 09:06:27.166455 19895 ts_manager.cc:210] Registered new tserver with Master: 5cb229095048484c972cfa2de0e574a9 (127.16.187.131:36919)
I20261117 09:06:27.166978 19895 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.131:38663
W20261117 09:06:27.241701 20349 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:27.241878 20349 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:27.241899 20349 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:27.241920 20349 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:27.243616 20349 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:27.243669 20349 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:27.243691 20349 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:27.243721 20349 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:27.243737 20349 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.132
I20261117 09:06:27.245432 20349 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.16.187.132
--webserver_port=0
--tserver_master_addrs=127.16.187.190:40495
--never_fsync=true
--heap_profile_path=/tmp/kudu.20349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:27.245656 20349 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:27.245826 20349 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:27.248216 20357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:27.248214 20355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:27.248275 20349 server_base.cc:1061] running on GCE node
W20261117 09:06:27.248219 20354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:27.248540 20349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:27.248733 20349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:27.249882 20349 hybrid_clock.cc:648] HybridClock initialized: now 1794906387249860 us; error 33 us; skew 500 ppm
I20261117 09:06:27.251072 20349 webserver.cc:577] Webserver started at http://127.16.187.132:37501/ using document root <none> and password file <none>
I20261117 09:06:27.251268 20349 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:27.251308 20349 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:27.251407 20349 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:27.252194 20349 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "58bb0c3dc9b540139356ea4f40c41b97"
format_stamp: "Formatted at 2026-11-17 09:06:27 on dist-test-slave-x52s"
I20261117 09:06:27.252538 20349 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "58bb0c3dc9b540139356ea4f40c41b97"
format_stamp: "Formatted at 2026-11-17 09:06:27 on dist-test-slave-x52s"
I20261117 09:06:27.253651 20349 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:27.254277 20363 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:27.254444 20349 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:27.254509 20349 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "58bb0c3dc9b540139356ea4f40c41b97"
format_stamp: "Formatted at 2026-11-17 09:06:27 on dist-test-slave-x52s"
I20261117 09:06:27.254590 20349 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:27.270273 20349 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:27.270485 20349 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:27.270614 20349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:27.270785 20349 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:27.271082 20349 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:27.271113 20349 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:27.271142 20349 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:27.271159 20349 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:27.278388 20349 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.132:42105
I20261117 09:06:27.278447 20476 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.132:42105 every 8 connection(s)
I20261117 09:06:27.278752 20349 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261117 09:06:27.280529 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 20349
I20261117 09:06:27.280630 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20261117 09:06:27.283188 20477 heartbeater.cc:345] Connected to a master server at 127.16.187.190:40495
I20261117 09:06:27.283281 20477 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:27.283463 20477 heartbeater.cc:508] Master 127.16.187.190:40495 requested a full tablet report, sending...
I20261117 09:06:27.283818 19895 ts_manager.cc:210] Registered new tserver with Master: 58bb0c3dc9b540139356ea4f40c41b97 (127.16.187.132:42105)
I20261117 09:06:27.284186 19895 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.132:41325
I20261117 09:06:27.292881 17134 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261117 09:06:27.298763 19895 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40356:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20261117 09:06:27.305675 20149 tablet_service.cc:1500] Processing CreateTablet for tablet cb8ca876149344a2b0cef892885865b0 (DEFAULT_TABLE table=TestTable [id=173b3ef7c376405c9fc1f700dad057f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:27.305974 20149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb8ca876149344a2b0cef892885865b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:27.306470 20411 tablet_service.cc:1500] Processing CreateTablet for tablet cb8ca876149344a2b0cef892885865b0 (DEFAULT_TABLE table=TestTable [id=173b3ef7c376405c9fc1f700dad057f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:27.306747 20411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb8ca876149344a2b0cef892885865b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:27.308283 20491 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Bootstrap starting.
I20261117 09:06:27.308957 20491 tablet_bootstrap.cc:654] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:27.308964 20492 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Bootstrap starting.
I20261117 09:06:27.309330 20491 log.cc:826] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:27.309348 20280 tablet_service.cc:1500] Processing CreateTablet for tablet cb8ca876149344a2b0cef892885865b0 (DEFAULT_TABLE table=TestTable [id=173b3ef7c376405c9fc1f700dad057f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:27.309500 20492 tablet_bootstrap.cc:654] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:27.309602 20280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb8ca876149344a2b0cef892885865b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:27.309763 20492 log.cc:826] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:27.309912 20491 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: No bootstrap required, opened a new log
I20261117 09:06:27.309975 20491 ts_tablet_manager.cc:1449] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:27.310334 20492 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: No bootstrap required, opened a new log
I20261117 09:06:27.310402 20492 ts_tablet_manager.cc:1449] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:27.311518 20492 raft_consensus.cc:360] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:27.311625 20492 raft_consensus.cc:386] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:27.311550 20491 raft_consensus.cc:360] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:27.311650 20492 raft_consensus.cc:748] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58bb0c3dc9b540139356ea4f40c41b97, State: Initialized, Role: FOLLOWER
I20261117 09:06:27.311666 20491 raft_consensus.cc:386] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:27.311691 20491 raft_consensus.cc:748] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd3880772e74fc0aff725468109a033, State: Initialized, Role: FOLLOWER
I20261117 09:06:27.311765 20492 consensus_queue.cc:261] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:27.311774 20491 consensus_queue.cc:261] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:27.311985 20492 ts_tablet_manager.cc:1480] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:27.312155 20497 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Bootstrap starting.
I20261117 09:06:27.312278 20477 heartbeater.cc:500] Master 127.16.187.190:40495 was elected leader, sending a full tablet report...
I20261117 09:06:27.312515 20491 ts_tablet_manager.cc:1480] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:27.312626 20215 heartbeater.cc:500] Master 127.16.187.190:40495 was elected leader, sending a full tablet report...
I20261117 09:06:27.312747 20497 tablet_bootstrap.cc:654] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:27.313004 20497 log.cc:826] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:27.313578 20497 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: No bootstrap required, opened a new log
I20261117 09:06:27.313642 20497 ts_tablet_manager.cc:1449] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:27.314762 20497 raft_consensus.cc:360] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:27.314874 20497 raft_consensus.cc:386] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:27.314891 20497 raft_consensus.cc:748] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb229095048484c972cfa2de0e574a9, State: Initialized, Role: FOLLOWER
I20261117 09:06:27.314968 20497 consensus_queue.cc:261] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:27.315167 20497 ts_tablet_manager.cc:1480] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:27.315217 20346 heartbeater.cc:500] Master 127.16.187.190:40495 was elected leader, sending a full tablet report...
W20261117 09:06:27.346490 20216 tablet.cc:2441] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261117 09:06:27.362290 20347 tablet.cc:2441] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261117 09:06:27.379215 20478 tablet.cc:2441] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261117 09:06:27.441151 20495 raft_consensus.cc:499] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261117 09:06:27.441277 20495 raft_consensus.cc:521] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:27.441581 20495 leader_election.cc:290] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fd3880772e74fc0aff725468109a033 (127.16.187.130:39781), 5cb229095048484c972cfa2de0e574a9 (127.16.187.131:36919)
I20261117 09:06:27.444940 20169 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cb8ca876149344a2b0cef892885865b0" candidate_uuid: "58bb0c3dc9b540139356ea4f40c41b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd3880772e74fc0aff725468109a033" is_pre_election: true
I20261117 09:06:27.445106 20169 raft_consensus.cc:2490] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58bb0c3dc9b540139356ea4f40c41b97 in term 0.
I20261117 09:06:27.445127 20300 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cb8ca876149344a2b0cef892885865b0" candidate_uuid: "58bb0c3dc9b540139356ea4f40c41b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb229095048484c972cfa2de0e574a9" is_pre_election: true
I20261117 09:06:27.445288 20300 raft_consensus.cc:2490] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58bb0c3dc9b540139356ea4f40c41b97 in term 0.
I20261117 09:06:27.445366 20364 leader_election.cc:304] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd3880772e74fc0aff725468109a033, 58bb0c3dc9b540139356ea4f40c41b97; no voters:
I20261117 09:06:27.445511 20495 raft_consensus.cc:2826] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261117 09:06:27.445565 20495 raft_consensus.cc:499] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261117 09:06:27.445587 20495 raft_consensus.cc:3096] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:27.446208 20495 raft_consensus.cc:521] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:27.446331 20495 leader_election.cc:290] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [CANDIDATE]: Term 1 election: Requested vote from peers 0fd3880772e74fc0aff725468109a033 (127.16.187.130:39781), 5cb229095048484c972cfa2de0e574a9 (127.16.187.131:36919)
I20261117 09:06:27.446552 20169 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cb8ca876149344a2b0cef892885865b0" candidate_uuid: "58bb0c3dc9b540139356ea4f40c41b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd3880772e74fc0aff725468109a033"
I20261117 09:06:27.446576 20300 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cb8ca876149344a2b0cef892885865b0" candidate_uuid: "58bb0c3dc9b540139356ea4f40c41b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb229095048484c972cfa2de0e574a9"
I20261117 09:06:27.446645 20300 raft_consensus.cc:3096] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:27.446642 20169 raft_consensus.cc:3096] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:27.447602 20300 raft_consensus.cc:2490] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58bb0c3dc9b540139356ea4f40c41b97 in term 1.
I20261117 09:06:27.447615 20169 raft_consensus.cc:2490] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58bb0c3dc9b540139356ea4f40c41b97 in term 1.
I20261117 09:06:27.447757 20364 leader_election.cc:304] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58bb0c3dc9b540139356ea4f40c41b97, 5cb229095048484c972cfa2de0e574a9; no voters:
I20261117 09:06:27.447853 20495 raft_consensus.cc:2826] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:27.448001 20495 raft_consensus.cc:705] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 1 LEADER]: Becoming Leader. State: Replica: 58bb0c3dc9b540139356ea4f40c41b97, State: Running, Role: LEADER
I20261117 09:06:27.448081 20495 consensus_queue.cc:238] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:27.448899 19896 catalog_manager.cc:5840] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58bb0c3dc9b540139356ea4f40c41b97 (127.16.187.132). New cstate: current_term: 1 leader_uuid: "58bb0c3dc9b540139356ea4f40c41b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } health_report { overall_health: UNKNOWN } } }
I20261117 09:06:27.449102 20503 consensus_queue.cc:1049] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:27.449203 20495 consensus_queue.cc:1049] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:27.487236 17134 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261117 09:06:27.487732 17134 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 0fd3880772e74fc0aff725468109a033 to finish bootstrapping
I20261117 09:06:27.491062 17134 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 5cb229095048484c972cfa2de0e574a9 to finish bootstrapping
I20261117 09:06:27.494050 17134 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 58bb0c3dc9b540139356ea4f40c41b97 to finish bootstrapping
I20261117 09:06:27.497184 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 20087
I20261117 09:06:27.510552 17134 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20261117 09:06:27.510810 17134 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
W20261117 09:06:27.517817 20364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.187.130:39781: connect: Connection refused (error 111)
W20261117 09:06:27.518307 20364 consensus_peers.cc:597] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 -> Peer 0fd3880772e74fc0aff725468109a033 (127.16.187.130:39781): Couldn't send request to peer 0fd3880772e74fc0aff725468109a033. Status: Network error: Client connection negotiation failed: client connection to 127.16.187.130:39781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261117 09:06:27.562796 17134 raft_consensus-itest-base.cc:274] Waiting for log GC on 58bb0c3dc9b540139356ea4f40c41b97
I20261117 09:06:27.562901 20347 maintenance_manager.cc:419] P 5cb229095048484c972cfa2de0e574a9: Scheduling FlushMRSOp(cb8ca876149344a2b0cef892885865b0): 5654038 bytes log retention, and flush 7160073 bytes memory
I20261117 09:06:27.567703 20237 maintenance_manager.cc:643] P 5cb229095048484c972cfa2de0e574a9: FlushMRSOp(cb8ca876149344a2b0cef892885865b0) complete. Timing: real 0.005s user 0.004s sys 0.000s Metrics: {"bytes_written":136065,"cfile_init":1,"compiler_manager_pool.queue_time_us":116,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":172,"dirs.run_wall_time_us":962,"drs_written":1,"lbm_read_time_us":19,"lbm_reads_lt_1ms":4,"lbm_write_time_us":188,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":47,"thread_start_us":74,"threads_started":1}
I20261117 09:06:27.567878 20347 maintenance_manager.cc:419] P 5cb229095048484c972cfa2de0e574a9: Scheduling LogGCOp(cb8ca876149344a2b0cef892885865b0): free 4076240 bytes of WAL
I20261117 09:06:27.567956 20237 log_reader.cc:385] T cb8ca876149344a2b0cef892885865b0: removed 5 log segments from log reader
I20261117 09:06:27.567986 20237 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000001 (ops 1-7)
I20261117 09:06:27.568011 20237 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000002 (ops 8-13)
I20261117 09:06:27.568025 20237 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000003 (ops 14-20)
I20261117 09:06:27.568046 20237 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000004 (ops 21-26)
I20261117 09:06:27.568069 20237 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000005 (ops 27-32)
I20261117 09:06:27.568840 20237 maintenance_manager.cc:643] P 5cb229095048484c972cfa2de0e574a9: LogGCOp(cb8ca876149344a2b0cef892885865b0) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261117 09:06:27.568959 20347 maintenance_manager.cc:419] P 5cb229095048484c972cfa2de0e574a9: Scheduling UndoDeltaBlockGCOp(cb8ca876149344a2b0cef892885865b0): 661 bytes on disk
I20261117 09:06:27.569106 20237 maintenance_manager.cc:643] P 5cb229095048484c972cfa2de0e574a9: UndoDeltaBlockGCOp(cb8ca876149344a2b0cef892885865b0) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":18,"lbm_reads_lt_1ms":4}
I20261117 09:06:27.579849 20478 maintenance_manager.cc:419] P 58bb0c3dc9b540139356ea4f40c41b97: Scheduling FlushMRSOp(cb8ca876149344a2b0cef892885865b0): 5522696 bytes log retention, and flush 7160073 bytes memory
I20261117 09:06:27.586158 20368 maintenance_manager.cc:643] P 58bb0c3dc9b540139356ea4f40c41b97: FlushMRSOp(cb8ca876149344a2b0cef892885865b0) complete. Timing: real 0.006s user 0.005s sys 0.000s Metrics: {"bytes_written":136065,"cfile_init":1,"compiler_manager_pool.queue_time_us":201,"dirs.queue_time_us":55,"dirs.run_cpu_time_us":167,"dirs.run_wall_time_us":1014,"drs_written":1,"lbm_read_time_us":19,"lbm_reads_lt_1ms":4,"lbm_write_time_us":277,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":47,"thread_start_us":121,"threads_started":1}
I20261117 09:06:27.586365 20478 maintenance_manager.cc:419] P 58bb0c3dc9b540139356ea4f40c41b97: Scheduling LogGCOp(cb8ca876149344a2b0cef892885865b0): free 3681746 bytes of WAL
I20261117 09:06:27.586463 20368 log_reader.cc:385] T cb8ca876149344a2b0cef892885865b0: removed 4 log segments from log reader
I20261117 09:06:27.586496 20368 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000001 (ops 1-8)
I20261117 09:06:27.586517 20368 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000002 (ops 9-15)
I20261117 09:06:27.586581 20368 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000003 (ops 16-22)
I20261117 09:06:27.586597 20368 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000004 (ops 23-29)
I20261117 09:06:27.587594 20368 maintenance_manager.cc:643] P 58bb0c3dc9b540139356ea4f40c41b97: LogGCOp(cb8ca876149344a2b0cef892885865b0) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261117 09:06:27.587733 20478 maintenance_manager.cc:419] P 58bb0c3dc9b540139356ea4f40c41b97: Scheduling UndoDeltaBlockGCOp(cb8ca876149344a2b0cef892885865b0): 661 bytes on disk
I20261117 09:06:27.587896 20368 maintenance_manager.cc:643] P 58bb0c3dc9b540139356ea4f40c41b97: UndoDeltaBlockGCOp(cb8ca876149344a2b0cef892885865b0) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":19,"lbm_reads_lt_1ms":4}
I20261117 09:06:27.593560 17134 raft_consensus-itest-base.cc:279] Log GC complete on 58bb0c3dc9b540139356ea4f40c41b97
I20261117 09:06:27.940529 20084 heartbeater.cc:500] Master 127.16.187.190:40495 was elected leader, sending a full tablet report...
I20261117 09:06:28.031563 20503 consensus_queue.cc:754] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [LEADER]: The logs necessary to catch up peer 0fd3880772e74fc0aff725468109a033 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 3..42: Segment 1 which contained index 3 has been GCed)
I20261117 09:06:28.031646 20503 consensus_queue.cc:580] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [LEADER]: The logs necessary to catch up peer 0fd3880772e74fc0aff725468109a033 have been garbage collected. The replica will never be able to catch up
I20261117 09:06:28.032711 20431 consensus_queue.cc:238] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 1.48, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:28.033217 20300 raft_consensus.cc:1285] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [term 1 FOLLOWER]: Refusing update from remote peer 58bb0c3dc9b540139356ea4f40c41b97: Log matching property violated. Preceding OpId in replica: term: 1 index: 48. Preceding OpId from leader: term: 1 index: 49. (index mismatch)
I20261117 09:06:28.033444 20495 consensus_queue.cc:1049] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20261117 09:06:28.034214 20503 raft_consensus.cc:2983] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 1 LEADER]: Committing config change with OpId 1.49: config changed from index -1 to 49, VOTER 0fd3880772e74fc0aff725468109a033 (127.16.187.130) evicted. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } } }
I20261117 09:06:28.034354 20300 raft_consensus.cc:2983] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [term 1 FOLLOWER]: Committing config change with OpId 1.49: config changed from index -1 to 49, VOTER 0fd3880772e74fc0aff725468109a033 (127.16.187.130) evicted. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } } }
I20261117 09:06:28.035319 19880 catalog_manager.cc:5247] ChangeConfig:REMOVE_PEER RPC for tablet cb8ca876149344a2b0cef892885865b0 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20261117 09:06:28.035473 19895 catalog_manager.cc:5840] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 reported cstate change: config changed from index -1 to 49, VOTER 0fd3880772e74fc0aff725468109a033 (127.16.187.130) evicted. New cstate: current_term: 1 leader_uuid: "58bb0c3dc9b540139356ea4f40c41b97" committed_config { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } } }
W20261117 09:06:28.036586 19880 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.187.130:39781: connect: Connection refused (error 111)
I20261117 09:06:28.036818 20431 consensus_queue.cc:238] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 49, Committed index: 49, Last appended: 1.49, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } } peers { permanent_uuid: "1290a67ac40240b0842588e83fc93605" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 34971 } attrs { promote: true } }
W20261117 09:06:28.037029 19880 catalog_manager.cc:4792] TS 0fd3880772e74fc0aff725468109a033 (127.16.187.130:39781): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cb8ca876149344a2b0cef892885865b0: Network error: Client connection negotiation failed: client connection to 127.16.187.130:39781: connect: Connection refused (error 111)
I20261117 09:06:28.037263 20300 raft_consensus.cc:1285] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [term 1 FOLLOWER]: Refusing update from remote peer 58bb0c3dc9b540139356ea4f40c41b97: Log matching property violated. Preceding OpId in replica: term: 1 index: 49. Preceding OpId from leader: term: 1 index: 50. (index mismatch)
I20261117 09:06:28.037446 20501 consensus_queue.cc:1049] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 50, Last known committed idx: 49, Time since last communication: 0.000s
I20261117 09:06:28.038105 20495 raft_consensus.cc:2983] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 [term 1 LEADER]: Committing config change with OpId 1.50: config changed from index 49 to 50, NON_VOTER 1290a67ac40240b0842588e83fc93605 (127.16.187.129) added. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } } peers { permanent_uuid: "1290a67ac40240b0842588e83fc93605" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 34971 } attrs { promote: true } } }
I20261117 09:06:28.038230 20300 raft_consensus.cc:2983] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9 [term 1 FOLLOWER]: Committing config change with OpId 1.50: config changed from index 49 to 50, NON_VOTER 1290a67ac40240b0842588e83fc93605 (127.16.187.129) added. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } } peers { permanent_uuid: "1290a67ac40240b0842588e83fc93605" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 34971 } attrs { promote: true } } }
I20261117 09:06:28.038931 19880 catalog_manager.cc:5247] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet cb8ca876149344a2b0cef892885865b0 with cas_config_opid_index 49: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20261117 09:06:28.039299 19896 catalog_manager.cc:5840] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 reported cstate change: config changed from index 49 to 50, NON_VOTER 1290a67ac40240b0842588e83fc93605 (127.16.187.129) added. New cstate: current_term: 1 leader_uuid: "58bb0c3dc9b540139356ea4f40c41b97" committed_config { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1290a67ac40240b0842588e83fc93605" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 34971 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20261117 09:06:28.041225 20365 consensus_peers.cc:597] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97 -> Peer 1290a67ac40240b0842588e83fc93605 (127.16.187.129:34971): Couldn't send request to peer 1290a67ac40240b0842588e83fc93605. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb8ca876149344a2b0cef892885865b0. This is attempt 1: this message will repeat every 5th retry.
I20261117 09:06:28.655228 20540 ts_tablet_manager.cc:979] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: Initiating tablet copy from peer 58bb0c3dc9b540139356ea4f40c41b97 (127.16.187.132:42105)
I20261117 09:06:28.655452 20540 tablet_copy_client.cc:323] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: tablet copy: Beginning tablet copy session from remote peer at address 127.16.187.132:42105
I20261117 09:06:28.658218 20451 tablet_copy_service.cc:140] P 58bb0c3dc9b540139356ea4f40c41b97: Received BeginTabletCopySession request for tablet cb8ca876149344a2b0cef892885865b0 from peer 1290a67ac40240b0842588e83fc93605 ({username='slave'} at 127.16.187.129:52175)
I20261117 09:06:28.658300 20451 tablet_copy_service.cc:161] P 58bb0c3dc9b540139356ea4f40c41b97: Beginning new tablet copy session on tablet cb8ca876149344a2b0cef892885865b0 from peer 1290a67ac40240b0842588e83fc93605 at {username='slave'} at 127.16.187.129:52175: session id = 1290a67ac40240b0842588e83fc93605-cb8ca876149344a2b0cef892885865b0
I20261117 09:06:28.658955 20451 tablet_copy_source_session.cc:215] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Tablet Copy: opened 5 blocks and 3 log segments
I20261117 09:06:28.659776 20540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb8ca876149344a2b0cef892885865b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:28.661621 20540 tablet_copy_client.cc:806] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: tablet copy: Starting download of 5 data blocks...
I20261117 09:06:28.665228 20540 tablet_copy_client.cc:670] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: tablet copy: Starting download of 3 WAL segments...
I20261117 09:06:28.671429 20540 tablet_copy_client.cc:538] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261117 09:06:28.672233 20540 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: Bootstrap starting.
I20261117 09:06:28.673928 20540 log.cc:826] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:28.676625 20540 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: Bootstrap replayed 1/3 log segments. Stats: ops{read=7 overwritten=0 applied=5 ignored=5} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=3. Pending: 2 replicates
I20261117 09:06:28.678297 20540 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: Bootstrap replayed 2/3 log segments. Stats: ops{read=14 overwritten=0 applied=12 ignored=12} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=3. Pending: 2 replicates
I20261117 09:06:28.679736 20540 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: Bootstrap replayed 3/3 log segments. Stats: ops{read=21 overwritten=0 applied=21 ignored=19} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=3. Pending: 0 replicates
I20261117 09:06:28.679941 20540 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: Bootstrap complete.
I20261117 09:06:28.680413 20540 ts_tablet_manager.cc:1449] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20261117 09:06:28.681140 20540 raft_consensus.cc:360] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } } peers { permanent_uuid: "1290a67ac40240b0842588e83fc93605" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 34971 } attrs { promote: true } }
I20261117 09:06:28.681260 20540 raft_consensus.cc:748] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1290a67ac40240b0842588e83fc93605, State: Initialized, Role: LEARNER
I20261117 09:06:28.681305 20540 consensus_queue.cc:261] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 50, Last appended: 1.50, Last appended by leader: 50, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } } peers { permanent_uuid: "1290a67ac40240b0842588e83fc93605" member_type: NON_VOTER last_known_addr { host: "127.16.187.129" port: 34971 } attrs { promote: true } }
I20261117 09:06:28.681499 20540 ts_tablet_manager.cc:1480] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20261117 09:06:28.681918 20443 tablet_copy_service.cc:342] P 58bb0c3dc9b540139356ea4f40c41b97: Request end of tablet copy session 1290a67ac40240b0842588e83fc93605-cb8ca876149344a2b0cef892885865b0 received from {username='slave'} at 127.16.187.129:52175
I20261117 09:06:28.681980 20443 tablet_copy_service.cc:434] P 58bb0c3dc9b540139356ea4f40c41b97: ending tablet copy session 1290a67ac40240b0842588e83fc93605-cb8ca876149344a2b0cef892885865b0 on tablet cb8ca876149344a2b0cef892885865b0 with peer 1290a67ac40240b0842588e83fc93605
I20261117 09:06:28.736392 20085 maintenance_manager.cc:419] P 1290a67ac40240b0842588e83fc93605: Scheduling UndoDeltaBlockGCOp(cb8ca876149344a2b0cef892885865b0): 661 bytes on disk
I20261117 09:06:28.736773 19975 maintenance_manager.cc:643] P 1290a67ac40240b0842588e83fc93605: UndoDeltaBlockGCOp(cb8ca876149344a2b0cef892885865b0) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":19,"lbm_reads_lt_1ms":4}
I20261117 09:06:29.180403 20038 raft_consensus.cc:1227] T cb8ca876149344a2b0cef892885865b0 P 1290a67ac40240b0842588e83fc93605 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.49->[1.50-1.50] Dedup: 1.50->[]
I20261117 09:06:29.191550 20478 maintenance_manager.cc:419] P 58bb0c3dc9b540139356ea4f40c41b97: Scheduling LogGCOp(cb8ca876149344a2b0cef892885865b0): free 1840950 bytes of WAL
I20261117 09:06:29.191738 20368 log_reader.cc:385] T cb8ca876149344a2b0cef892885865b0: removed 2 log segments from log reader
I20261117 09:06:29.191788 20368 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000005 (ops 30-36)
I20261117 09:06:29.191809 20368 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 58bb0c3dc9b540139356ea4f40c41b97: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000006 (ops 37-43)
I20261117 09:06:29.192260 20368 maintenance_manager.cc:643] P 58bb0c3dc9b540139356ea4f40c41b97: LogGCOp(cb8ca876149344a2b0cef892885865b0) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
W20261117 09:06:29.244719 19880 catalog_manager.cc:4792] TS 0fd3880772e74fc0aff725468109a033 (127.16.187.130:39781): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cb8ca876149344a2b0cef892885865b0: Network error: Client connection negotiation failed: client connection to 127.16.187.130:39781: connect: Connection refused (error 111)
I20261117 09:06:29.594378 17134 raft_consensus-itest-base.cc:295] Servers converged with original term 1
I20261117 09:06:29.594445 17134 raft_consensus-itest-base.cc:303] Restarting 0fd3880772e74fc0aff725468109a033
I20261117 09:06:29.594666 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.130:39781
--local_ip_for_outbound_sockets=127.16.187.130
--tserver_master_addrs=127.16.187.190:40495
--webserver_port=45037
--webserver_interface=127.16.187.130
--builtin_ntp_servers=127.16.187.148:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261117 09:06:29.669261 20551 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:29.669431 20551 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:29.669451 20551 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:29.669476 20551 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:29.671080 20551 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:29.671128 20551 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:29.671147 20551 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:29.671175 20551 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:29.671191 20551 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.130
I20261117 09:06:29.672808 20551 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.130:39781
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.16.187.130
--webserver_port=45037
--tserver_master_addrs=127.16.187.190:40495
--never_fsync=true
--heap_profile_path=/tmp/kudu.20551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:29.673017 20551 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:29.673180 20551 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20261117 09:06:29.673576 20347 maintenance_manager.cc:419] P 5cb229095048484c972cfa2de0e574a9: Scheduling LogGCOp(cb8ca876149344a2b0cef892885865b0): free 1577798 bytes of WAL
I20261117 09:06:29.673713 20237 log_reader.cc:385] T cb8ca876149344a2b0cef892885865b0: removed 2 log segments from log reader
I20261117 09:06:29.673751 20237 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000006 (ops 33-38)
I20261117 09:06:29.673774 20237 log.cc:1079] T cb8ca876149344a2b0cef892885865b0 P 5cb229095048484c972cfa2de0e574a9: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb8ca876149344a2b0cef892885865b0/wal-000000007 (ops 39-44)
I20261117 09:06:29.674248 20237 maintenance_manager.cc:643] P 5cb229095048484c972cfa2de0e574a9: LogGCOp(cb8ca876149344a2b0cef892885865b0) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
W20261117 09:06:29.675906 20556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:29.675978 20557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:29.676029 20559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:29.676112 20551 server_base.cc:1061] running on GCE node
I20261117 09:06:29.676244 20551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:29.676390 20551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:29.677515 20551 hybrid_clock.cc:648] HybridClock initialized: now 1794906389677494 us; error 40 us; skew 500 ppm
I20261117 09:06:29.678597 20551 webserver.cc:577] Webserver started at http://127.16.187.130:45037/ using document root <none> and password file <none>
I20261117 09:06:29.678763 20551 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:29.678821 20551 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:29.679914 20551 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20261117 09:06:29.680528 20565 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:29.680701 20551 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261117 09:06:29.680755 20551 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0fd3880772e74fc0aff725468109a033"
format_stamp: "Formatted at 2026-11-17 09:06:27 on dist-test-slave-x52s"
I20261117 09:06:29.681056 20551 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:29.698438 20551 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:29.698709 20551 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:29.698825 20551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:29.699023 20551 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:29.699431 20572 ts_tablet_manager.cc:588] Loading tablet metadata (0/1 complete)
I20261117 09:06:29.700217 20551 ts_tablet_manager.cc:631] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20261117 09:06:29.700253 20551 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20261117 09:06:29.700286 20551 ts_tablet_manager.cc:646] Registering tablets (0/1 complete)
I20261117 09:06:29.700824 20551 ts_tablet_manager.cc:662] Registered 1 tablets
I20261117 09:06:29.700853 20551 ts_tablet_manager.cc:641] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20261117 09:06:29.700947 20572 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Bootstrap starting.
I20261117 09:06:29.704149 20572 log.cc:826] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:29.705282 20572 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261117 09:06:29.705570 20572 tablet_bootstrap.cc:492] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Bootstrap complete.
I20261117 09:06:29.705834 20572 ts_tablet_manager.cc:1449] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20261117 09:06:29.706610 20551 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.130:39781
I20261117 09:06:29.706745 20680 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.130:39781 every 8 connection(s)
I20261117 09:06:29.706957 20551 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261117 09:06:29.707620 20572 raft_consensus.cc:360] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:29.707881 20572 raft_consensus.cc:748] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd3880772e74fc0aff725468109a033, State: Initialized, Role: FOLLOWER
I20261117 09:06:29.708014 20572 consensus_queue.cc:261] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd3880772e74fc0aff725468109a033" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 39781 } } peers { permanent_uuid: "58bb0c3dc9b540139356ea4f40c41b97" member_type: VOTER last_known_addr { host: "127.16.187.132" port: 42105 } } peers { permanent_uuid: "5cb229095048484c972cfa2de0e574a9" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 36919 } }
I20261117 09:06:29.708227 20572 ts_tablet_manager.cc:1480] T cb8ca876149344a2b0cef892885865b0 P 0fd3880772e74fc0aff725468109a033: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:29.709410 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 20551
I20261117 09:06:29.711242 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 19956
I20261117 09:06:29.715739 20681 heartbeater.cc:345] Connected to a master server at 127.16.187.190:40495
I20261117 09:06:29.715853 20681 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:29.716084 20681 heartbeater.cc:508] Master 127.16.187.190:40495 requested a full tablet report, sending...
I20261117 09:06:29.716665 19896 ts_manager.cc:210] Re-registered known tserver with Master: 0fd3880772e74fc0aff725468109a033 (127.16.187.130:39781)
I20261117 09:06:29.717111 19896 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.130:39455
I20261117 09:06:29.717193 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 20551
W20261117 09:06:29.721067 19880 connection.cc:573] server connection from 127.16.187.130:39455 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
I20261117 09:06:29.721254 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 20218
I20261117 09:06:29.726250 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 20349
I20261117 09:06:29.731215 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 19865
2026-11-17T09:06:29Z chronyd exiting
[ OK ] SlowTabletBootstrapTest.TestFallBehind (3075 ms)
[ RUN ] SlowTabletBootstrapTest.TestFallBehindSlowBootstrap
I20261117 09:06:29.751767 17134 ts_itest-base.cc:115] Starting cluster with:
I20261117 09:06:29.751811 17134 ts_itest-base.cc:116] --------------
I20261117 09:06:29.751827 17134 ts_itest-base.cc:117] 4 tablet servers
I20261117 09:06:29.751838 17134 ts_itest-base.cc:118] 3 replicas per TS
I20261117 09:06:29.751852 17134 ts_itest-base.cc:119] --------------
2026-11-17T09:06:29Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-17T09:06:29Z Disabled control of system clock
I20261117 09:06:29.758180 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:38681
--webserver_interface=127.16.187.190
--webserver_port=0
--builtin_ntp_servers=127.16.187.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.187.190:38681 with env {}
W20261117 09:06:29.834259 20693 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:29.834402 20693 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:29.834426 20693 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:29.835923 20693 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261117 09:06:29.835969 20693 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:29.835987 20693 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261117 09:06:29.836002 20693 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261117 09:06:29.837523 20693 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.187.190:38681
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.187.190:38681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.16.187.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.20693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:29.837705 20693 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:29.837860 20693 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:29.840031 20701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:29.840154 20693 server_base.cc:1061] running on GCE node
W20261117 09:06:29.840111 20698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:29.840070 20699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:29.840443 20693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:29.840678 20693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:29.841841 20693 hybrid_clock.cc:648] HybridClock initialized: now 1794906389841831 us; error 27 us; skew 500 ppm
I20261117 09:06:29.842877 20693 webserver.cc:577] Webserver started at http://127.16.187.190:33729/ using document root <none> and password file <none>
I20261117 09:06:29.843039 20693 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:29.843077 20693 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:29.843163 20693 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:29.843947 20693 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "8b04ce0921ba4148914ea18aff808ad5"
format_stamp: "Formatted at 2026-11-17 09:06:29 on dist-test-slave-x52s"
I20261117 09:06:29.844239 20693 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "8b04ce0921ba4148914ea18aff808ad5"
format_stamp: "Formatted at 2026-11-17 09:06:29 on dist-test-slave-x52s"
I20261117 09:06:29.845310 20693 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261117 09:06:29.845918 20707 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:29.846072 20693 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261117 09:06:29.846128 20693 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
uuid: "8b04ce0921ba4148914ea18aff808ad5"
format_stamp: "Formatted at 2026-11-17 09:06:29 on dist-test-slave-x52s"
I20261117 09:06:29.846182 20693 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:29.862145 20693 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:29.862382 20693 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:29.862491 20693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:29.865883 20693 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.190:38681
I20261117 09:06:29.865916 20759 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.190:38681 every 8 connection(s)
I20261117 09:06:29.866202 20693 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20261117 09:06:29.866740 20760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:29.868778 20760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5: Bootstrap starting.
I20261117 09:06:29.869411 20760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:29.869623 20760 log.cc:826] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:29.870136 20760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5: No bootstrap required, opened a new log
I20261117 09:06:29.871337 20760 raft_consensus.cc:360] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b04ce0921ba4148914ea18aff808ad5" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38681 } }
I20261117 09:06:29.871441 20760 raft_consensus.cc:386] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:29.871459 20760 raft_consensus.cc:748] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b04ce0921ba4148914ea18aff808ad5, State: Initialized, Role: FOLLOWER
I20261117 09:06:29.871548 20760 consensus_queue.cc:261] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b04ce0921ba4148914ea18aff808ad5" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38681 } }
I20261117 09:06:29.871611 20760 raft_consensus.cc:400] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261117 09:06:29.871635 20760 raft_consensus.cc:499] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261117 09:06:29.871658 20760 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:29.871795 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 20693
I20261117 09:06:29.871878 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/master-0/wal/instance
I20261117 09:06:29.872177 20760 raft_consensus.cc:521] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b04ce0921ba4148914ea18aff808ad5" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38681 } }
I20261117 09:06:29.872269 20760 leader_election.cc:304] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b04ce0921ba4148914ea18aff808ad5; no voters:
I20261117 09:06:29.872397 20760 leader_election.cc:290] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261117 09:06:29.872462 20763 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:29.872570 20763 raft_consensus.cc:705] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [term 1 LEADER]: Becoming Leader. State: Replica: 8b04ce0921ba4148914ea18aff808ad5, State: Running, Role: LEADER
I20261117 09:06:29.872608 20760 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [sys.catalog]: configured and running, proceeding with master startup.
I20261117 09:06:29.872645 20763 consensus_queue.cc:238] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b04ce0921ba4148914ea18aff808ad5" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38681 } }
I20261117 09:06:29.872920 20763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b04ce0921ba4148914ea18aff808ad5. Latest consensus state: current_term: 1 leader_uuid: "8b04ce0921ba4148914ea18aff808ad5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b04ce0921ba4148914ea18aff808ad5" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38681 } } }
I20261117 09:06:29.872931 20765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b04ce0921ba4148914ea18aff808ad5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b04ce0921ba4148914ea18aff808ad5" member_type: VOTER last_known_addr { host: "127.16.187.190" port: 38681 } } }
I20261117 09:06:29.873011 20763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:29.873049 20765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5 [sys.catalog]: This master's current role is: LEADER
I20261117 09:06:29.873198 20770 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261117 09:06:29.873673 20770 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261117 09:06:29.875362 20770 catalog_manager.cc:1398] Generated new cluster ID: 69d06362473b4441ba46fb12ecddb504
I20261117 09:06:29.875411 20770 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261117 09:06:29.888667 20770 catalog_manager.cc:1421] Generated new certificate authority record
I20261117 09:06:29.889219 20770 catalog_manager.cc:1555] Loading token signing keys...
I20261117 09:06:29.894440 20770 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 8b04ce0921ba4148914ea18aff808ad5: Generated new TSK 0
I20261117 09:06:29.894635 20770 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261117 09:06:29.900756 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.129:0
--local_ip_for_outbound_sockets=127.16.187.129
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38681
--builtin_ntp_servers=127.16.187.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261117 09:06:29.977344 20784 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:29.977507 20784 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:29.977520 20784 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:29.977536 20784 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:29.979283 20784 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:29.979336 20784 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:29.979351 20784 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:29.979374 20784 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:29.979387 20784 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.129
I20261117 09:06:29.981058 20784 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.16.187.129
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38681
--never_fsync=true
--heap_profile_path=/tmp/kudu.20784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:29.981238 20784 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:29.981403 20784 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:29.983706 20789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:29.983705 20792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:29.983719 20790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:29.984256 20784 server_base.cc:1061] running on GCE node
I20261117 09:06:29.984400 20784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:29.984565 20784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:29.985723 20784 hybrid_clock.cc:648] HybridClock initialized: now 1794906389985709 us; error 34 us; skew 500 ppm
I20261117 09:06:29.987018 20784 webserver.cc:577] Webserver started at http://127.16.187.129:43117/ using document root <none> and password file <none>
I20261117 09:06:29.987215 20784 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:29.987258 20784 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:29.987336 20784 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:29.988134 20784 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "5dfb540aecee48009ff83f95b3af6cfc"
format_stamp: "Formatted at 2026-11-17 09:06:29 on dist-test-slave-x52s"
I20261117 09:06:29.988436 20784 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "5dfb540aecee48009ff83f95b3af6cfc"
format_stamp: "Formatted at 2026-11-17 09:06:29 on dist-test-slave-x52s"
I20261117 09:06:29.989585 20784 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:29.990295 20798 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:29.990473 20784 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:29.990582 20784 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "5dfb540aecee48009ff83f95b3af6cfc"
format_stamp: "Formatted at 2026-11-17 09:06:29 on dist-test-slave-x52s"
I20261117 09:06:29.990643 20784 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:30.007194 20784 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:30.007452 20784 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:30.007570 20784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:30.007771 20784 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:30.008121 20784 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:30.008165 20784 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.008203 20784 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:30.008225 20784 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.014909 20784 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.129:44181
I20261117 09:06:30.014940 20911 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.129:44181 every 8 connection(s)
I20261117 09:06:30.015316 20784 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20261117 09:06:30.019881 20912 heartbeater.cc:345] Connected to a master server at 127.16.187.190:38681
I20261117 09:06:30.020004 20912 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:30.020187 20912 heartbeater.cc:508] Master 127.16.187.190:38681 requested a full tablet report, sending...
I20261117 09:06:30.020675 20724 ts_manager.cc:210] Registered new tserver with Master: 5dfb540aecee48009ff83f95b3af6cfc (127.16.187.129:44181)
I20261117 09:06:30.021330 20724 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.129:60759
I20261117 09:06:30.025525 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 20784
I20261117 09:06:30.025626 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20261117 09:06:30.026746 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.130:0
--local_ip_for_outbound_sockets=127.16.187.130
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38681
--builtin_ntp_servers=127.16.187.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261117 09:06:30.101028 20915 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:30.101169 20915 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:30.101182 20915 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:30.101197 20915 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:30.102733 20915 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:30.102773 20915 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:30.102788 20915 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:30.102809 20915 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:30.102833 20915 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.130
I20261117 09:06:30.104381 20915 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.16.187.130
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38681
--never_fsync=true
--heap_profile_path=/tmp/kudu.20915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:30.104555 20915 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:30.104724 20915 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:30.107045 20923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:30.107065 20921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:30.107156 20915 server_base.cc:1061] running on GCE node
W20261117 09:06:30.107090 20920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:30.107401 20915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:30.107571 20915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:30.108711 20915 hybrid_clock.cc:648] HybridClock initialized: now 1794906390108699 us; error 27 us; skew 500 ppm
I20261117 09:06:30.109845 20915 webserver.cc:577] Webserver started at http://127.16.187.130:38337/ using document root <none> and password file <none>
I20261117 09:06:30.110035 20915 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:30.110075 20915 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:30.110163 20915 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:30.111034 20915 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "07cf9c6dcb4f46a6b3d506c4903fd589"
format_stamp: "Formatted at 2026-11-17 09:06:30 on dist-test-slave-x52s"
I20261117 09:06:30.111334 20915 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "07cf9c6dcb4f46a6b3d506c4903fd589"
format_stamp: "Formatted at 2026-11-17 09:06:30 on dist-test-slave-x52s"
I20261117 09:06:30.112556 20915 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:30.113477 20929 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.113643 20915 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:30.113714 20915 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "07cf9c6dcb4f46a6b3d506c4903fd589"
format_stamp: "Formatted at 2026-11-17 09:06:30 on dist-test-slave-x52s"
I20261117 09:06:30.113775 20915 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:30.137529 20915 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:30.137732 20915 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:30.137820 20915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:30.137980 20915 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:30.138301 20915 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:30.138334 20915 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.138355 20915 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:30.138368 20915 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.143493 20915 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.130:36641
I20261117 09:06:30.143553 21042 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.130:36641 every 8 connection(s)
I20261117 09:06:30.143837 20915 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261117 09:06:30.147964 21043 heartbeater.cc:345] Connected to a master server at 127.16.187.190:38681
I20261117 09:06:30.148059 21043 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:30.148239 21043 heartbeater.cc:508] Master 127.16.187.190:38681 requested a full tablet report, sending...
I20261117 09:06:30.148618 20724 ts_manager.cc:210] Registered new tserver with Master: 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641)
I20261117 09:06:30.148993 20724 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.130:48173
I20261117 09:06:30.151125 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 20915
I20261117 09:06:30.151212 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20261117 09:06:30.152197 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.131:0
--local_ip_for_outbound_sockets=127.16.187.131
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38681
--builtin_ntp_servers=127.16.187.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261117 09:06:30.227898 21046 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:30.228068 21046 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:30.228082 21046 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:30.228098 21046 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:30.229640 21046 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:30.229688 21046 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:30.229703 21046 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:30.229724 21046 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:30.229735 21046 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.131
I20261117 09:06:30.231361 21046 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.16.187.131
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38681
--never_fsync=true
--heap_profile_path=/tmp/kudu.21046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:30.231534 21046 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:30.231675 21046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:30.234050 21054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:30.234150 21051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:30.234218 21046 server_base.cc:1061] running on GCE node
W20261117 09:06:30.234052 21052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:30.234395 21046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:30.234598 21046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:30.235724 21046 hybrid_clock.cc:648] HybridClock initialized: now 1794906390235714 us; error 23 us; skew 500 ppm
I20261117 09:06:30.236742 21046 webserver.cc:577] Webserver started at http://127.16.187.131:44247/ using document root <none> and password file <none>
I20261117 09:06:30.236913 21046 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:30.236953 21046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:30.237069 21046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:30.237856 21046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "66a005b3d40c4ea4be5b9e204f6693d6"
format_stamp: "Formatted at 2026-11-17 09:06:30 on dist-test-slave-x52s"
I20261117 09:06:30.238165 21046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "66a005b3d40c4ea4be5b9e204f6693d6"
format_stamp: "Formatted at 2026-11-17 09:06:30 on dist-test-slave-x52s"
I20261117 09:06:30.239351 21046 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261117 09:06:30.240164 21060 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.240330 21046 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261117 09:06:30.240388 21046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "66a005b3d40c4ea4be5b9e204f6693d6"
format_stamp: "Formatted at 2026-11-17 09:06:30 on dist-test-slave-x52s"
I20261117 09:06:30.240446 21046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:30.261976 21046 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:30.262228 21046 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:30.262336 21046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:30.262585 21046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:30.262902 21046 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:30.262933 21046 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.262962 21046 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:30.263020 21046 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.268371 21046 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.131:35145
I20261117 09:06:30.268417 21173 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.131:35145 every 8 connection(s)
I20261117 09:06:30.268726 21046 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261117 09:06:30.273144 21174 heartbeater.cc:345] Connected to a master server at 127.16.187.190:38681
I20261117 09:06:30.273257 21174 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:30.273440 21174 heartbeater.cc:508] Master 127.16.187.190:38681 requested a full tablet report, sending...
I20261117 09:06:30.273797 20724 ts_manager.cc:210] Registered new tserver with Master: 66a005b3d40c4ea4be5b9e204f6693d6 (127.16.187.131:35145)
I20261117 09:06:30.274148 20724 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.131:55503
I20261117 09:06:30.276911 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 21046
I20261117 09:06:30.277000 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20261117 09:06:30.278106 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.132:0
--local_ip_for_outbound_sockets=127.16.187.132
--webserver_interface=127.16.187.132
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38681
--builtin_ntp_servers=127.16.187.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261117 09:06:30.356884 21177 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:30.357060 21177 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:30.357074 21177 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:30.357092 21177 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:30.358848 21177 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:30.358902 21177 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:30.358918 21177 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:30.358945 21177 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:30.358973 21177 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.132
I20261117 09:06:30.360682 21177 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.16.187.132
--webserver_port=0
--tserver_master_addrs=127.16.187.190:38681
--never_fsync=true
--heap_profile_path=/tmp/kudu.21177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:30.360898 21177 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:30.361078 21177 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:30.363493 21183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:30.363493 21182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:30.363510 21185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:30.363642 21177 server_base.cc:1061] running on GCE node
I20261117 09:06:30.363828 21177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:30.364037 21177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:30.365191 21177 hybrid_clock.cc:648] HybridClock initialized: now 1794906390365173 us; error 29 us; skew 500 ppm
I20261117 09:06:30.366427 21177 webserver.cc:577] Webserver started at http://127.16.187.132:42895/ using document root <none> and password file <none>
I20261117 09:06:30.366643 21177 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:30.366691 21177 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:30.366786 21177 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261117 09:06:30.367623 21177 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "7bbb4a7125364035adf4449e956e814d"
format_stamp: "Formatted at 2026-11-17 09:06:30 on dist-test-slave-x52s"
I20261117 09:06:30.367946 21177 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "7bbb4a7125364035adf4449e956e814d"
format_stamp: "Formatted at 2026-11-17 09:06:30 on dist-test-slave-x52s"
I20261117 09:06:30.369097 21177 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261117 09:06:30.369837 21191 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.370007 21177 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:30.370075 21177 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "7bbb4a7125364035adf4449e956e814d"
format_stamp: "Formatted at 2026-11-17 09:06:30 on dist-test-slave-x52s"
I20261117 09:06:30.370131 21177 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:30.420444 21177 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:30.420665 21177 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:30.420773 21177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:30.420950 21177 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:30.421267 21177 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261117 09:06:30.421298 21177 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.421329 21177 ts_tablet_manager.cc:662] Registered 0 tablets
I20261117 09:06:30.421345 21177 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.426468 21177 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.132:46437
I20261117 09:06:30.426561 21304 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.132:46437 every 8 connection(s)
I20261117 09:06:30.426822 21177 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261117 09:06:30.430675 21305 heartbeater.cc:345] Connected to a master server at 127.16.187.190:38681
I20261117 09:06:30.430763 21305 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:30.430917 21305 heartbeater.cc:508] Master 127.16.187.190:38681 requested a full tablet report, sending...
I20261117 09:06:30.431278 20724 ts_manager.cc:210] Registered new tserver with Master: 7bbb4a7125364035adf4449e956e814d (127.16.187.132:46437)
I20261117 09:06:30.431612 20724 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.132:50015
I20261117 09:06:30.433228 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 21177
I20261117 09:06:30.433316 17134 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20261117 09:06:30.434391 17134 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261117 09:06:30.439721 20724 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47010:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20261117 09:06:30.447280 20846 tablet_service.cc:1500] Processing CreateTablet for tablet 994bc462e617467fa3f469d33cea9954 (DEFAULT_TABLE table=TestTable [id=907f47957291433284789641ac58a2cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:30.447273 20977 tablet_service.cc:1500] Processing CreateTablet for tablet 994bc462e617467fa3f469d33cea9954 (DEFAULT_TABLE table=TestTable [id=907f47957291433284789641ac58a2cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:30.447571 20846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 994bc462e617467fa3f469d33cea9954. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:30.447572 20977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 994bc462e617467fa3f469d33cea9954. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:30.448112 21108 tablet_service.cc:1500] Processing CreateTablet for tablet 994bc462e617467fa3f469d33cea9954 (DEFAULT_TABLE table=TestTable [id=907f47957291433284789641ac58a2cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20261117 09:06:30.448336 21108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 994bc462e617467fa3f469d33cea9954. 1 dirs total, 0 dirs full, 0 dirs failed
I20261117 09:06:30.450122 21323 tablet_bootstrap.cc:492] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Bootstrap starting.
I20261117 09:06:30.450220 21324 tablet_bootstrap.cc:492] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Bootstrap starting.
I20261117 09:06:30.450660 21322 tablet_bootstrap.cc:492] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: Bootstrap starting.
I20261117 09:06:30.450878 21323 tablet_bootstrap.cc:654] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:30.450887 21324 tablet_bootstrap.cc:654] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:30.451120 21323 log.cc:826] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:30.451139 21324 log.cc:826] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:30.451249 21322 tablet_bootstrap.cc:654] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: Neither blocks nor log segments found. Creating new log.
I20261117 09:06:30.451534 21322 log.cc:826] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:30.451628 21323 tablet_bootstrap.cc:492] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: No bootstrap required, opened a new log
I20261117 09:06:30.451690 21323 ts_tablet_manager.cc:1449] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20261117 09:06:30.452078 21322 tablet_bootstrap.cc:492] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: No bootstrap required, opened a new log
I20261117 09:06:30.452078 21324 tablet_bootstrap.cc:492] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: No bootstrap required, opened a new log
I20261117 09:06:30.452142 21324 ts_tablet_manager.cc:1449] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20261117 09:06:30.452142 21322 ts_tablet_manager.cc:1449] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261117 09:06:30.452913 21323 raft_consensus.cc:360] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:30.453019 21323 raft_consensus.cc:386] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:30.453043 21323 raft_consensus.cc:748] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cf9c6dcb4f46a6b3d506c4903fd589, State: Initialized, Role: FOLLOWER
I20261117 09:06:30.453150 21323 consensus_queue.cc:261] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:30.453439 21323 ts_tablet_manager.cc:1480] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261117 09:06:30.453579 21043 heartbeater.cc:500] Master 127.16.187.190:38681 was elected leader, sending a full tablet report...
I20261117 09:06:30.453621 21324 raft_consensus.cc:360] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:30.453622 21322 raft_consensus.cc:360] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:30.453734 21322 raft_consensus.cc:386] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:30.453734 21324 raft_consensus.cc:386] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261117 09:06:30.453758 21322 raft_consensus.cc:748] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dfb540aecee48009ff83f95b3af6cfc, State: Initialized, Role: FOLLOWER
I20261117 09:06:30.453759 21324 raft_consensus.cc:748] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a005b3d40c4ea4be5b9e204f6693d6, State: Initialized, Role: FOLLOWER
I20261117 09:06:30.453838 21322 consensus_queue.cc:261] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:30.453838 21324 consensus_queue.cc:261] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:30.454053 21324 ts_tablet_manager.cc:1480] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261117 09:06:30.454053 21322 ts_tablet_manager.cc:1480] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261117 09:06:30.454154 20912 heartbeater.cc:500] Master 127.16.187.190:38681 was elected leader, sending a full tablet report...
I20261117 09:06:30.454173 21174 heartbeater.cc:500] Master 127.16.187.190:38681 was elected leader, sending a full tablet report...
W20261117 09:06:30.469372 21175 tablet.cc:2441] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261117 09:06:30.516530 20913 tablet.cc:2441] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261117 09:06:30.544768 21044 tablet.cc:2441] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261117 09:06:30.685765 21328 raft_consensus.cc:499] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261117 09:06:30.685909 21328 raft_consensus.cc:521] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:30.686249 21328 leader_election.cc:290] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dfb540aecee48009ff83f95b3af6cfc (127.16.187.129:44181), 66a005b3d40c4ea4be5b9e204f6693d6 (127.16.187.131:35145)
I20261117 09:06:30.689674 20866 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dfb540aecee48009ff83f95b3af6cfc" is_pre_election: true
I20261117 09:06:30.689848 20866 raft_consensus.cc:2490] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07cf9c6dcb4f46a6b3d506c4903fd589 in term 0.
I20261117 09:06:30.689889 21128 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" is_pre_election: true
I20261117 09:06:30.690011 21128 raft_consensus.cc:2490] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07cf9c6dcb4f46a6b3d506c4903fd589 in term 0.
I20261117 09:06:30.690100 20931 leader_election.cc:304] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cf9c6dcb4f46a6b3d506c4903fd589, 5dfb540aecee48009ff83f95b3af6cfc; no voters:
I20261117 09:06:30.690231 21328 raft_consensus.cc:2826] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261117 09:06:30.690297 21328 raft_consensus.cc:499] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261117 09:06:30.690315 21328 raft_consensus.cc:3096] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:30.690968 21328 raft_consensus.cc:521] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:30.691083 21328 leader_election.cc:290] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [CANDIDATE]: Term 1 election: Requested vote from peers 5dfb540aecee48009ff83f95b3af6cfc (127.16.187.129:44181), 66a005b3d40c4ea4be5b9e204f6693d6 (127.16.187.131:35145)
I20261117 09:06:30.691284 20866 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dfb540aecee48009ff83f95b3af6cfc"
I20261117 09:06:30.691362 20866 raft_consensus.cc:3096] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:30.691329 21128 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a005b3d40c4ea4be5b9e204f6693d6"
I20261117 09:06:30.691412 21128 raft_consensus.cc:3096] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 0 FOLLOWER]: Advancing to term 1
I20261117 09:06:30.691941 20866 raft_consensus.cc:2490] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cf9c6dcb4f46a6b3d506c4903fd589 in term 1.
I20261117 09:06:30.692126 20931 leader_election.cc:304] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cf9c6dcb4f46a6b3d506c4903fd589, 5dfb540aecee48009ff83f95b3af6cfc; no voters:
I20261117 09:06:30.692276 21328 raft_consensus.cc:2826] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 1 FOLLOWER]: Leader election won for term 1
I20261117 09:06:30.692458 21328 raft_consensus.cc:705] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 1 LEADER]: Becoming Leader. State: Replica: 07cf9c6dcb4f46a6b3d506c4903fd589, State: Running, Role: LEADER
I20261117 09:06:30.692546 21128 raft_consensus.cc:2490] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cf9c6dcb4f46a6b3d506c4903fd589 in term 1.
I20261117 09:06:30.692553 21328 consensus_queue.cc:238] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:30.693567 21333 consensus_queue.cc:1049] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:30.693544 20722 catalog_manager.cc:5840] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130). New cstate: current_term: 1 leader_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } health_report { overall_health: HEALTHY } } }
I20261117 09:06:30.693771 21335 consensus_queue.cc:1049] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261117 09:06:30.729669 17134 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261117 09:06:30.730170 17134 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 5dfb540aecee48009ff83f95b3af6cfc to finish bootstrapping
I20261117 09:06:30.733673 17134 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 07cf9c6dcb4f46a6b3d506c4903fd589 to finish bootstrapping
I20261117 09:06:30.736727 17134 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 66a005b3d40c4ea4be5b9e204f6693d6 to finish bootstrapping
I20261117 09:06:30.739538 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 20915
I20261117 09:06:30.745024 17134 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskkXwYHv/build/release/bin/kudu
/tmp/dist-test-taskkXwYHv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.187.130:36641
--local_ip_for_outbound_sockets=127.16.187.130
--tserver_master_addrs=127.16.187.190:38681
--webserver_port=38337
--webserver_interface=127.16.187.130
--builtin_ntp_servers=127.16.187.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000
--tablet_bootstrap_inject_latency_ms=3000 with env {}
W20261117 09:06:30.820752 21342 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261117 09:06:30.820946 21342 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261117 09:06:30.820968 21342 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261117 09:06:30.820997 21342 flags.cc:432] Enabled unsafe flag: --tablet_bootstrap_inject_latency_ms=3000
W20261117 09:06:30.821024 21342 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261117 09:06:30.822768 21342 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261117 09:06:30.822834 21342 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261117 09:06:30.822860 21342 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261117 09:06:30.822896 21342 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261117 09:06:30.822926 21342 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.187.130
I20261117 09:06:30.824651 21342 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.187.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.187.130:36641
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.16.187.130
--webserver_port=38337
--tserver_master_addrs=127.16.187.190:38681
--tablet_bootstrap_inject_latency_ms=3000
--never_fsync=true
--heap_profile_path=/tmp/kudu.21342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.187.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 17 Nov 2026 08:43:15 UTC on e7f111948823
build id 14824
I20261117 09:06:30.824882 21342 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261117 09:06:30.825067 21342 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261117 09:06:30.827543 21348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:30.827541 21347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261117 09:06:30.827541 21350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261117 09:06:30.827862 21342 server_base.cc:1061] running on GCE node
I20261117 09:06:30.828065 21342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261117 09:06:30.828251 21342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261117 09:06:30.829401 21342 hybrid_clock.cc:648] HybridClock initialized: now 1794906390829386 us; error 27 us; skew 500 ppm
I20261117 09:06:30.830499 21342 webserver.cc:577] Webserver started at http://127.16.187.130:38337/ using document root <none> and password file <none>
I20261117 09:06:30.830706 21342 fs_manager.cc:362] Metadata directory not provided
I20261117 09:06:30.830766 21342 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261117 09:06:30.831880 21342 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20261117 09:06:30.832508 21356 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261117 09:06:30.832681 21342 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261117 09:06:30.832738 21342 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "07cf9c6dcb4f46a6b3d506c4903fd589"
format_stamp: "Formatted at 2026-11-17 09:06:30 on dist-test-slave-x52s"
I20261117 09:06:30.833000 21342 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261117 09:06:30.841130 21342 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261117 09:06:30.841333 21342 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261117 09:06:30.841427 21342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261117 09:06:30.841609 21342 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261117 09:06:30.842001 21363 ts_tablet_manager.cc:588] Loading tablet metadata (0/1 complete)
I20261117 09:06:30.842864 21342 ts_tablet_manager.cc:631] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20261117 09:06:30.842911 21342 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20261117 09:06:30.842945 21342 ts_tablet_manager.cc:646] Registering tablets (0/1 complete)
I20261117 09:06:30.843457 21342 ts_tablet_manager.cc:662] Registered 1 tablets
W20261117 09:06:30.843495 21363 ts_tablet_manager.cc:1428] Injecting 3000ms delay in tablet bootstrapping
I20261117 09:06:30.843502 21342 ts_tablet_manager.cc:641] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20261117 09:06:30.849566 21342 rpc_server.cc:307] RPC server started. Bound to: 127.16.187.130:36641
I20261117 09:06:30.849622 21470 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.187.130:36641 every 8 connection(s)
I20261117 09:06:30.849918 21342 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261117 09:06:30.854658 21471 heartbeater.cc:345] Connected to a master server at 127.16.187.190:38681
I20261117 09:06:30.854760 21471 heartbeater.cc:462] Registering TS with master...
I20261117 09:06:30.854961 21471 heartbeater.cc:508] Master 127.16.187.190:38681 requested a full tablet report, sending...
I20261117 09:06:30.855545 20724 ts_manager.cc:210] Re-registered known tserver with Master: 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641)
I20261117 09:06:30.856007 20724 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.16.187.130:57603
I20261117 09:06:30.859465 17134 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskkXwYHv/build/release/bin/kudu as pid 21342
I20261117 09:06:31.432317 21305 heartbeater.cc:500] Master 127.16.187.190:38681 was elected leader, sending a full tablet report...
I20261117 09:06:31.856828 21471 heartbeater.cc:500] Master 127.16.187.190:38681 was elected leader, sending a full tablet report...
I20261117 09:06:32.196208 21476 raft_consensus.cc:499] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 07cf9c6dcb4f46a6b3d506c4903fd589)
I20261117 09:06:32.196300 21476 raft_consensus.cc:521] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:32.196448 21477 raft_consensus.cc:499] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 07cf9c6dcb4f46a6b3d506c4903fd589)
I20261117 09:06:32.196522 21477 raft_consensus.cc:521] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:32.196794 21477 leader_election.cc:290] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66a005b3d40c4ea4be5b9e204f6693d6 (127.16.187.131:35145), 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641)
I20261117 09:06:32.196903 21476 leader_election.cc:290] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5dfb540aecee48009ff83f95b3af6cfc (127.16.187.129:44181), 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641)
I20261117 09:06:32.200445 20866 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5dfb540aecee48009ff83f95b3af6cfc" is_pre_election: true
I20261117 09:06:32.200495 21128 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "5dfb540aecee48009ff83f95b3af6cfc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" is_pre_election: true
I20261117 09:06:32.200546 20866 raft_consensus.cc:2490] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a005b3d40c4ea4be5b9e204f6693d6 in term 1.
I20261117 09:06:32.200557 21128 raft_consensus.cc:2490] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dfb540aecee48009ff83f95b3af6cfc in term 1.
I20261117 09:06:32.200747 21062 leader_election.cc:304] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dfb540aecee48009ff83f95b3af6cfc, 66a005b3d40c4ea4be5b9e204f6693d6; no voters:
I20261117 09:06:32.200757 20800 leader_election.cc:304] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dfb540aecee48009ff83f95b3af6cfc, 66a005b3d40c4ea4be5b9e204f6693d6; no voters:
I20261117 09:06:32.200872 21477 raft_consensus.cc:2826] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261117 09:06:32.200910 21477 raft_consensus.cc:499] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 07cf9c6dcb4f46a6b3d506c4903fd589)
I20261117 09:06:32.200927 21477 raft_consensus.cc:3096] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 1 FOLLOWER]: Advancing to term 2
I20261117 09:06:32.200937 21476 raft_consensus.cc:2826] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261117 09:06:32.200975 21476 raft_consensus.cc:499] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 07cf9c6dcb4f46a6b3d506c4903fd589)
I20261117 09:06:32.200996 21476 raft_consensus.cc:3096] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 1 FOLLOWER]: Advancing to term 2
I20261117 09:06:32.200965 21425 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "5dfb540aecee48009ff83f95b3af6cfc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" is_pre_election: true
I20261117 09:06:32.200999 21424 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" is_pre_election: true
W20261117 09:06:32.202036 20799 leader_election.cc:343] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641): Illegal state: must be running to vote when last-logged opid is not known
I20261117 09:06:32.203704 21477 raft_consensus.cc:521] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:32.203718 21476 raft_consensus.cc:521] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:32.203864 21476 leader_election.cc:290] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [CANDIDATE]: Term 2 election: Requested vote from peers 5dfb540aecee48009ff83f95b3af6cfc (127.16.187.129:44181), 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641)
I20261117 09:06:32.203877 21477 leader_election.cc:290] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 2 election: Requested vote from peers 66a005b3d40c4ea4be5b9e204f6693d6 (127.16.187.131:35145), 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641)
I20261117 09:06:32.204062 21425 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589"
I20261117 09:06:32.204082 21128 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "5dfb540aecee48009ff83f95b3af6cfc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66a005b3d40c4ea4be5b9e204f6693d6"
I20261117 09:06:32.204108 20866 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5dfb540aecee48009ff83f95b3af6cfc"
I20261117 09:06:32.204116 21424 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "5dfb540aecee48009ff83f95b3af6cfc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589"
I20261117 09:06:32.204156 21128 raft_consensus.cc:2415] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5dfb540aecee48009ff83f95b3af6cfc in current term 2: Already voted for candidate 66a005b3d40c4ea4be5b9e204f6693d6 in this term.
I20261117 09:06:32.204180 20866 raft_consensus.cc:2415] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 66a005b3d40c4ea4be5b9e204f6693d6 in current term 2: Already voted for candidate 5dfb540aecee48009ff83f95b3af6cfc in this term.
W20261117 09:06:32.204295 20799 leader_election.cc:343] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641): Illegal state: must be running to vote when last-logged opid is not known
W20261117 09:06:32.204360 21061 leader_election.cc:343] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641): Illegal state: must be running to vote when last-logged opid is not known
I20261117 09:06:32.204427 21061 leader_election.cc:304] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66a005b3d40c4ea4be5b9e204f6693d6; no voters: 07cf9c6dcb4f46a6b3d506c4903fd589, 5dfb540aecee48009ff83f95b3af6cfc
I20261117 09:06:32.204452 20800 leader_election.cc:304] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5dfb540aecee48009ff83f95b3af6cfc; no voters: 07cf9c6dcb4f46a6b3d506c4903fd589, 66a005b3d40c4ea4be5b9e204f6693d6
I20261117 09:06:32.204520 21476 raft_consensus.cc:2771] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20261117 09:06:32.204530 21477 raft_consensus.cc:2771] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20261117 09:06:33.798681 21486 raft_consensus.cc:499] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261117 09:06:33.798779 21486 raft_consensus.cc:521] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:33.798981 21486 leader_election.cc:290] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a005b3d40c4ea4be5b9e204f6693d6 (127.16.187.131:35145), 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641)
I20261117 09:06:33.799242 21128 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "5dfb540aecee48009ff83f95b3af6cfc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" is_pre_election: true
I20261117 09:06:33.799232 21424 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "5dfb540aecee48009ff83f95b3af6cfc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" is_pre_election: true
I20261117 09:06:33.799337 21128 raft_consensus.cc:2490] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dfb540aecee48009ff83f95b3af6cfc in term 2.
I20261117 09:06:33.799499 20800 leader_election.cc:304] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dfb540aecee48009ff83f95b3af6cfc, 66a005b3d40c4ea4be5b9e204f6693d6; no voters:
W20261117 09:06:33.799571 20799 leader_election.cc:343] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641): Illegal state: must be running to vote when last-logged opid is not known
I20261117 09:06:33.799614 21486 raft_consensus.cc:2826] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20261117 09:06:33.799645 21486 raft_consensus.cc:499] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261117 09:06:33.799669 21486 raft_consensus.cc:3096] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 2 FOLLOWER]: Advancing to term 3
I20261117 09:06:33.800276 21486 raft_consensus.cc:521] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:33.800402 21486 leader_election.cc:290] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 3 election: Requested vote from peers 66a005b3d40c4ea4be5b9e204f6693d6 (127.16.187.131:35145), 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641)
I20261117 09:06:33.800571 21424 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "5dfb540aecee48009ff83f95b3af6cfc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589"
I20261117 09:06:33.800585 21128 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "994bc462e617467fa3f469d33cea9954" candidate_uuid: "5dfb540aecee48009ff83f95b3af6cfc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66a005b3d40c4ea4be5b9e204f6693d6"
I20261117 09:06:33.800650 21128 raft_consensus.cc:3096] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 2 FOLLOWER]: Advancing to term 3
W20261117 09:06:33.800786 20799 leader_election.cc:343] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641): Illegal state: must be running to vote when last-logged opid is not known
I20261117 09:06:33.801227 21128 raft_consensus.cc:2490] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dfb540aecee48009ff83f95b3af6cfc in term 3.
I20261117 09:06:33.801421 20800 leader_election.cc:304] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5dfb540aecee48009ff83f95b3af6cfc, 66a005b3d40c4ea4be5b9e204f6693d6; no voters: 07cf9c6dcb4f46a6b3d506c4903fd589
I20261117 09:06:33.801510 21486 raft_consensus.cc:2826] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 3 FOLLOWER]: Leader election won for term 3
I20261117 09:06:33.801627 21486 raft_consensus.cc:705] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [term 3 LEADER]: Becoming Leader. State: Replica: 5dfb540aecee48009ff83f95b3af6cfc, State: Running, Role: LEADER
I20261117 09:06:33.801719 21486 consensus_queue.cc:238] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:33.802348 21128 raft_consensus.cc:1285] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6 [term 3 FOLLOWER]: Refusing update from remote peer 5dfb540aecee48009ff83f95b3af6cfc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20261117 09:06:33.802417 20723 catalog_manager.cc:5840] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc reported cstate change: term changed from 1 to 3, leader changed from 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130) to 5dfb540aecee48009ff83f95b3af6cfc (127.16.187.129). New cstate: current_term: 3 leader_uuid: "5dfb540aecee48009ff83f95b3af6cfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } health_report { overall_health: UNKNOWN } } }
I20261117 09:06:33.802589 21489 consensus_queue.cc:1049] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20261117 09:06:33.802762 20799 consensus_peers.cc:597] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc -> Peer 07cf9c6dcb4f46a6b3d506c4903fd589 (127.16.187.130:36641): Couldn't send request to peer 07cf9c6dcb4f46a6b3d506c4903fd589. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20261117 09:06:33.843752 21363 tablet_bootstrap.cc:492] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Bootstrap starting.
I20261117 09:06:33.846503 21363 log.cc:826] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Log is configured to *not* fsync() on all Append() calls
I20261117 09:06:33.847332 21363 tablet_bootstrap.cc:492] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261117 09:06:33.847561 21363 tablet_bootstrap.cc:492] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Bootstrap complete.
I20261117 09:06:33.847749 21363 ts_tablet_manager.cc:1449] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20261117 09:06:33.848434 21363 raft_consensus.cc:360] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:33.848603 21363 raft_consensus.cc:748] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cf9c6dcb4f46a6b3d506c4903fd589, State: Initialized, Role: FOLLOWER
I20261117 09:06:33.848697 21363 consensus_queue.cc:261] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dfb540aecee48009ff83f95b3af6cfc" member_type: VOTER last_known_addr { host: "127.16.187.129" port: 44181 } } peers { permanent_uuid: "66a005b3d40c4ea4be5b9e204f6693d6" member_type: VOTER last_known_addr { host: "127.16.187.131" port: 35145 } } peers { permanent_uuid: "07cf9c6dcb4f46a6b3d506c4903fd589" member_type: VOTER last_known_addr { host: "127.16.187.130" port: 36641 } }
I20261117 09:06:33.848905 21363 ts_tablet_manager.cc:1480] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20261117 09:06:34.285236 21424 raft_consensus.cc:3096] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589 [term 1 FOLLOWER]: Advancing to term 3
I20261117 09:06:34.494369 17134 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20261117 09:06:34.494659 17134 raft_config_change-itest.cc:610] Waiting until we've written at least 4MB...
I20261117 09:06:34.525488 20913 maintenance_manager.cc:419] P 5dfb540aecee48009ff83f95b3af6cfc: Scheduling FlushMRSOp(994bc462e617467fa3f469d33cea9954): 2761400 bytes log retention, and flush 4038921 bytes memory
I20261117 09:06:34.535315 20803 maintenance_manager.cc:643] P 5dfb540aecee48009ff83f95b3af6cfc: FlushMRSOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.010s user 0.003s sys 0.003s Metrics: {"bytes_written":135869,"cfile_init":1,"compiler_manager_pool.queue_time_us":133,"dirs.queue_time_us":148,"drs_written":1,"lbm_read_time_us":18,"lbm_reads_lt_1ms":4,"lbm_write_time_us":294,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":20,"spinlock_wait_cycles":384,"thread_start_us":179,"threads_started":2}
I20261117 09:06:34.535665 20913 maintenance_manager.cc:419] P 5dfb540aecee48009ff83f95b3af6cfc: Scheduling LogGCOp(994bc462e617467fa3f469d33cea9954): free 1840906 bytes of WAL
I20261117 09:06:34.535784 20803 log_reader.cc:385] T 994bc462e617467fa3f469d33cea9954: removed 2 log segments from log reader
I20261117 09:06:34.535822 20803 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000001 (ops 1-9)
I20261117 09:06:34.535843 20803 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000002 (ops 10-16)
I20261117 09:06:34.536414 20803 maintenance_manager.cc:643] P 5dfb540aecee48009ff83f95b3af6cfc: LogGCOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261117 09:06:34.536533 20913 maintenance_manager.cc:419] P 5dfb540aecee48009ff83f95b3af6cfc: Scheduling FlushMRSOp(994bc462e617467fa3f469d33cea9954): 1840912 bytes log retention, and flush 1958153 bytes memory
I20261117 09:06:34.540307 20803 maintenance_manager.cc:643] P 5dfb540aecee48009ff83f95b3af6cfc: FlushMRSOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.004s user 0.001s sys 0.001s Metrics: {"bytes_written":135737,"cfile_init":1,"dirs.queue_time_us":2062,"drs_written":1,"lbm_read_time_us":431,"lbm_reads_lt_1ms":4,"lbm_write_time_us":412,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":8,"thread_start_us":110,"threads_started":1}
I20261117 09:06:34.540542 20913 maintenance_manager.cc:419] P 5dfb540aecee48009ff83f95b3af6cfc: Scheduling LogGCOp(994bc462e617467fa3f469d33cea9954): free 920494 bytes of WAL
I20261117 09:06:34.540657 20803 log_reader.cc:385] T 994bc462e617467fa3f469d33cea9954: removed 1 log segments from log reader
I20261117 09:06:34.540735 20803 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000003 (ops 17-23)
I20261117 09:06:34.541064 20803 maintenance_manager.cc:643] P 5dfb540aecee48009ff83f95b3af6cfc: LogGCOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20261117 09:06:34.541179 20913 maintenance_manager.cc:419] P 5dfb540aecee48009ff83f95b3af6cfc: Scheduling CompactRowSetsOp(994bc462e617467fa3f469d33cea9954): perf score=0.911778
I20261117 09:06:34.547775 17134 raft_config_change-itest.cc:616] Waiting for log GC on 5dfb540aecee48009ff83f95b3af6cfc
I20261117 09:06:34.547931 17134 raft_config_change-itest.cc:620] Log GC complete on 5dfb540aecee48009ff83f95b3af6cfc
I20261117 09:06:34.548909 20803 maintenance_manager.cc:643] P 5dfb540aecee48009ff83f95b3af6cfc: CompactRowSetsOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.008s user 0.004s sys 0.000s Metrics: {"bytes_written":135937,"cfile_cache_hit":10,"cfile_cache_hit_bytes":262391,"cfile_cache_miss":26,"cfile_cache_miss_bytes":263985,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":20,"dirs.run_cpu_time_us":145,"dirs.run_wall_time_us":1510,"drs_written":1,"lbm_read_time_us":430,"lbm_reads_lt_1ms":54,"lbm_write_time_us":281,"lbm_writes_lt_1ms":23,"num_input_rowsets":2,"peak_mem_usage":66799,"rows_written":28,"thread_start_us":221,"threads_started":4}
I20261117 09:06:34.549224 20913 maintenance_manager.cc:419] P 5dfb540aecee48009ff83f95b3af6cfc: Scheduling LogGCOp(994bc462e617467fa3f469d33cea9954): free 920418 bytes of WAL
I20261117 09:06:34.549321 20803 log_reader.cc:385] T 994bc462e617467fa3f469d33cea9954: removed 1 log segments from log reader
I20261117 09:06:34.549361 20803 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 5dfb540aecee48009ff83f95b3af6cfc: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-0/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000004 (ops 24-30)
I20261117 09:06:34.549603 20803 maintenance_manager.cc:643] P 5dfb540aecee48009ff83f95b3af6cfc: LogGCOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20261117 09:06:34.549757 20913 maintenance_manager.cc:419] P 5dfb540aecee48009ff83f95b3af6cfc: Scheduling UndoDeltaBlockGCOp(994bc462e617467fa3f469d33cea9954): 444 bytes on disk
I20261117 09:06:34.549921 20803 maintenance_manager.cc:643] P 5dfb540aecee48009ff83f95b3af6cfc: UndoDeltaBlockGCOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":19,"lbm_reads_lt_1ms":4}
I20261117 09:06:34.556681 21472 maintenance_manager.cc:419] P 07cf9c6dcb4f46a6b3d506c4903fd589: Scheduling FlushMRSOp(994bc462e617467fa3f469d33cea9954): 4865147 bytes log retention, and flush 6119689 bytes memory
I20261117 09:06:34.563524 21361 maintenance_manager.cc:643] P 07cf9c6dcb4f46a6b3d506c4903fd589: FlushMRSOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.007s user 0.000s sys 0.005s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":136,"dirs.queue_time_us":205,"dirs.run_cpu_time_us":136,"dirs.run_wall_time_us":713,"drs_written":1,"lbm_read_time_us":19,"lbm_reads_lt_1ms":4,"lbm_write_time_us":277,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":38,"thread_start_us":195,"threads_started":2}
I20261117 09:06:34.563726 21472 maintenance_manager.cc:419] P 07cf9c6dcb4f46a6b3d506c4903fd589: Scheduling LogGCOp(994bc462e617467fa3f469d33cea9954): free 4865147 bytes of WAL
I20261117 09:06:34.563809 21361 log_reader.cc:385] T 994bc462e617467fa3f469d33cea9954: removed 6 log segments from log reader
I20261117 09:06:34.563844 21361 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000001 (ops 1-8)
I20261117 09:06:34.563866 21361 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000002 (ops 9-14)
I20261117 09:06:34.563930 21361 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000003 (ops 15-21)
I20261117 09:06:34.563957 21361 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000004 (ops 22-26)
I20261117 09:06:34.564004 21361 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000005 (ops 27-32)
I20261117 09:06:34.564037 21361 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 07cf9c6dcb4f46a6b3d506c4903fd589: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-1/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000006 (ops 33-39)
I20261117 09:06:34.565374 21361 maintenance_manager.cc:643] P 07cf9c6dcb4f46a6b3d506c4903fd589: LogGCOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {}
I20261117 09:06:34.565536 21472 maintenance_manager.cc:419] P 07cf9c6dcb4f46a6b3d506c4903fd589: Scheduling UndoDeltaBlockGCOp(994bc462e617467fa3f469d33cea9954): 559 bytes on disk
I20261117 09:06:34.565711 21361 maintenance_manager.cc:643] P 07cf9c6dcb4f46a6b3d506c4903fd589: UndoDeltaBlockGCOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":21,"lbm_reads_lt_1ms":4}
I20261117 09:06:34.580215 21175 maintenance_manager.cc:419] P 66a005b3d40c4ea4be5b9e204f6693d6: Scheduling FlushMRSOp(994bc462e617467fa3f469d33cea9954): 4207671 bytes log retention, and flush 6119689 bytes memory
I20261117 09:06:34.587723 21065 maintenance_manager.cc:643] P 66a005b3d40c4ea4be5b9e204f6693d6: FlushMRSOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.007s user 0.005s sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":178,"drs_written":1,"lbm_read_time_us":19,"lbm_reads_lt_1ms":4,"lbm_write_time_us":280,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":38,"thread_start_us":229,"threads_started":2}
I20261117 09:06:34.587929 21175 maintenance_manager.cc:419] P 66a005b3d40c4ea4be5b9e204f6693d6: Scheduling LogGCOp(994bc462e617467fa3f469d33cea9954): free 4207671 bytes of WAL
I20261117 09:06:34.588075 21065 log_reader.cc:385] T 994bc462e617467fa3f469d33cea9954: removed 5 log segments from log reader
I20261117 09:06:34.588105 21065 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000001 (ops 1-8)
I20261117 09:06:34.588126 21065 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000002 (ops 9-14)
I20261117 09:06:34.588145 21065 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000003 (ops 15-20)
I20261117 09:06:34.588160 21065 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000004 (ops 21-27)
I20261117 09:06:34.588176 21065 log.cc:1079] T 994bc462e617467fa3f469d33cea9954 P 66a005b3d40c4ea4be5b9e204f6693d6: Deleting log segment in path: /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0/raft_consensus-itest-cluster/ts-2/wal/wals/994bc462e617467fa3f469d33cea9954/wal-000000005 (ops 28-34)
I20261117 09:06:34.589330 21065 maintenance_manager.cc:643] P 66a005b3d40c4ea4be5b9e204f6693d6: LogGCOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261117 09:06:34.589609 21175 maintenance_manager.cc:419] P 66a005b3d40c4ea4be5b9e204f6693d6: Scheduling UndoDeltaBlockGCOp(994bc462e617467fa3f469d33cea9954): 559 bytes on disk
I20261117 09:06:34.589833 21065 maintenance_manager.cc:643] P 66a005b3d40c4ea4be5b9e204f6693d6: UndoDeltaBlockGCOp(994bc462e617467fa3f469d33cea9954) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":21,"lbm_reads_lt_1ms":4}
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/integration-tests/raft_config_change-itest.cc:500: Failure
Value of: IsRaftConfigMember(added_replica_uuid, cstate.committed_config())
Actual: false
Expected: true
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20261117 09:07:04.633791 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 20784
I20261117 09:07:04.639192 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 21342
I20261117 09:07:04.644037 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 21046
I20261117 09:07:04.648517 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 21177
I20261117 09:07:04.652628 17134 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskkXwYHv/build/release/bin/kudu with pid 20693
2026-11-17T09:07:04Z chronyd exiting
I20261117 09:07:04.667757 17134 test_util.cc:182] -----------------------------------------------
I20261117 09:07:04.667845 17134 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskkXwYHv/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794906366961029-17134-0
[ FAILED ] SlowTabletBootstrapTest.TestFallBehindSlowBootstrap (34916 ms)
[----------] 3 tests from SlowTabletBootstrapTest (45986 ms total)
[----------] Global test environment tear-down
[==========] 6 tests from 2 test suites ran. (57702 ms total)
[ PASSED ] 5 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SlowTabletBootstrapTest.TestFallBehindSlowBootstrap
1 FAILED TEST