[==========] Running 6 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from RaftConfigChangeITest
[ RUN ] RaftConfigChangeITest.TestKudu2147
2026-11-18T03:06:15Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-18T03:06:15Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20261118 03:06:15.881516 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:42523
--webserver_interface=127.24.137.190
--webserver_port=0
--builtin_ntp_servers=127.24.137.148:46643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.137.190:42523 with env {}
W20261118 03:06:15.955629 25134 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:15.955788 25134 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:15.955806 25134 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:15.957329 25134 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 03:06:15.957374 25134 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:15.957387 25134 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 03:06:15.957397 25134 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 03:06:15.958992 25134 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:46643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.137.190:42523
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:42523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.137.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.25134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:15.959164 25134 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:15.959303 25134 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:15.961480 25140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:15.961508 25142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:15.961560 25134 server_base.cc:1061] running on GCE node
W20261118 03:06:15.961488 25139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:15.961925 25134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:15.962108 25134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:15.963225 25134 hybrid_clock.cc:648] HybridClock initialized: now 1794971175963213 us; error 30 us; skew 500 ppm
I20261118 03:06:15.964221 25134 webserver.cc:577] Webserver started at http://127.24.137.190:35529/ using document root <none> and password file <none>
I20261118 03:06:15.964381 25134 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:15.964417 25134 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:15.964504 25134 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:15.965402 25134 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/data/instance:
uuid: "0c664f0f7e86484998ab7fea222a7386"
format_stamp: "Formatted at 2026-11-18 03:06:15 on dist-test-slave-k60w"
I20261118 03:06:15.965776 25134 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/wal/instance:
uuid: "0c664f0f7e86484998ab7fea222a7386"
format_stamp: "Formatted at 2026-11-18 03:06:15 on dist-test-slave-k60w"
I20261118 03:06:15.966835 25134 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:15.967402 25148 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:15.967561 25134 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:15.967618 25134 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/wal
uuid: "0c664f0f7e86484998ab7fea222a7386"
format_stamp: "Formatted at 2026-11-18 03:06:15 on dist-test-slave-k60w"
I20261118 03:06:15.967672 25134 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:15.974014 25134 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:15.974237 25134 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:15.974330 25134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:15.977942 25134 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.190:42523
I20261118 03:06:15.977968 25200 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.190:42523 every 8 connection(s)
I20261118 03:06:15.978266 25134 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/data/info.pb
I20261118 03:06:15.978757 25201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:15.980808 25201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386: Bootstrap starting.
I20261118 03:06:15.981369 25201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:15.981623 25201 log.cc:826] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:15.982126 25201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386: No bootstrap required, opened a new log
I20261118 03:06:15.983283 25201 raft_consensus.cc:360] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c664f0f7e86484998ab7fea222a7386" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 42523 } }
I20261118 03:06:15.983403 25201 raft_consensus.cc:386] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:15.983430 25201 raft_consensus.cc:748] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c664f0f7e86484998ab7fea222a7386, State: Initialized, Role: FOLLOWER
I20261118 03:06:15.983536 25201 consensus_queue.cc:261] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c664f0f7e86484998ab7fea222a7386" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 42523 } }
I20261118 03:06:15.983585 25201 raft_consensus.cc:400] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261118 03:06:15.983605 25201 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261118 03:06:15.983644 25201 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:15.984128 25201 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c664f0f7e86484998ab7fea222a7386" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 42523 } }
I20261118 03:06:15.984225 25201 leader_election.cc:304] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c664f0f7e86484998ab7fea222a7386; no voters:
I20261118 03:06:15.984375 25201 leader_election.cc:290] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261118 03:06:15.984424 25204 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:15.984552 25204 raft_consensus.cc:705] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [term 1 LEADER]: Becoming Leader. State: Replica: 0c664f0f7e86484998ab7fea222a7386, State: Running, Role: LEADER
I20261118 03:06:15.984580 25201 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 03:06:15.984637 25204 consensus_queue.cc:238] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c664f0f7e86484998ab7fea222a7386" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 42523 } }
I20261118 03:06:15.984951 25205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c664f0f7e86484998ab7fea222a7386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c664f0f7e86484998ab7fea222a7386" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 42523 } } }
I20261118 03:06:15.984963 25206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c664f0f7e86484998ab7fea222a7386. Latest consensus state: current_term: 1 leader_uuid: "0c664f0f7e86484998ab7fea222a7386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c664f0f7e86484998ab7fea222a7386" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 42523 } } }
I20261118 03:06:15.985109 25206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [sys.catalog]: This master's current role is: LEADER
I20261118 03:06:15.985100 25205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386 [sys.catalog]: This master's current role is: LEADER
I20261118 03:06:15.985231 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 25134
I20261118 03:06:15.985328 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/master-0/wal/instance
I20261118 03:06:15.985411 25208 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 03:06:15.985944 25208 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 03:06:15.987656 25208 catalog_manager.cc:1398] Generated new cluster ID: df8555506cb94be79dbae2e0520561ad
I20261118 03:06:15.987711 25208 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 03:06:16.017426 25208 catalog_manager.cc:1421] Generated new certificate authority record
I20261118 03:06:16.017993 25208 catalog_manager.cc:1555] Loading token signing keys...
I20261118 03:06:16.023312 25208 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 0c664f0f7e86484998ab7fea222a7386: Generated new TSK 0
I20261118 03:06:16.023518 25208 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 03:06:16.026912 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.129:0
--local_ip_for_outbound_sockets=127.24.137.129
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:42523
--builtin_ntp_servers=127.24.137.148:46643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000 with env {}
W20261118 03:06:16.102526 25225 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:16.102679 25225 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:16.102706 25225 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:16.104303 25225 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:16.104370 25225 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.129
I20261118 03:06:16.106057 25225 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:46643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:42523
--never_fsync=true
--heap_profile_path=/tmp/kudu.25225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:16.106256 25225 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:16.106423 25225 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:16.108697 25231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:16.108705 25233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:16.108861 25225 server_base.cc:1061] running on GCE node
W20261118 03:06:16.108702 25230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:16.109068 25225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:16.109241 25225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:16.110356 25225 hybrid_clock.cc:648] HybridClock initialized: now 1794971176110348 us; error 34 us; skew 500 ppm
I20261118 03:06:16.111327 25225 webserver.cc:577] Webserver started at http://127.24.137.129:44373/ using document root <none> and password file <none>
I20261118 03:06:16.111505 25225 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:16.111546 25225 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:16.111632 25225 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:16.112398 25225 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/data/instance:
uuid: "30d7feb0bab14f308ec6c222be965cc3"
format_stamp: "Formatted at 2026-11-18 03:06:16 on dist-test-slave-k60w"
I20261118 03:06:16.112703 25225 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/wal/instance:
uuid: "30d7feb0bab14f308ec6c222be965cc3"
format_stamp: "Formatted at 2026-11-18 03:06:16 on dist-test-slave-k60w"
I20261118 03:06:16.113811 25225 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261118 03:06:16.114486 25239 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:16.114643 25225 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:16.114703 25225 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/wal
uuid: "30d7feb0bab14f308ec6c222be965cc3"
format_stamp: "Formatted at 2026-11-18 03:06:16 on dist-test-slave-k60w"
I20261118 03:06:16.114758 25225 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:16.123991 25225 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:16.124253 25225 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:16.124354 25225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:16.124539 25225 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:16.124828 25225 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:16.124857 25225 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:16.124886 25225 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:16.124909 25225 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:16.130362 25225 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.129:38305
I20261118 03:06:16.130421 25352 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.129:38305 every 8 connection(s)
I20261118 03:06:16.130705 25225 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/data/info.pb
I20261118 03:06:16.130859 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 25225
I20261118 03:06:16.130936 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-0/wal/instance
I20261118 03:06:16.132076 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.130:0
--local_ip_for_outbound_sockets=127.24.137.130
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:42523
--builtin_ntp_servers=127.24.137.148:46643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000 with env {}
I20261118 03:06:16.135857 25353 heartbeater.cc:345] Connected to a master server at 127.24.137.190:42523
I20261118 03:06:16.136009 25353 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:16.136236 25353 heartbeater.cc:508] Master 127.24.137.190:42523 requested a full tablet report, sending...
I20261118 03:06:16.136778 25165 ts_manager.cc:210] Registered new tserver with Master: 30d7feb0bab14f308ec6c222be965cc3 (127.24.137.129:38305)
I20261118 03:06:16.137856 25165 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.129:51845
W20261118 03:06:16.208695 25356 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:16.208854 25356 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:16.208873 25356 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:16.210448 25356 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:16.210516 25356 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.130
I20261118 03:06:16.212092 25356 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:46643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:42523
--never_fsync=true
--heap_profile_path=/tmp/kudu.25356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:16.212268 25356 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:16.212402 25356 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:16.214607 25364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:16.214617 25362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:16.214607 25361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:16.214879 25356 server_base.cc:1061] running on GCE node
I20261118 03:06:16.215001 25356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:16.215162 25356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:16.216287 25356 hybrid_clock.cc:648] HybridClock initialized: now 1794971176216274 us; error 31 us; skew 500 ppm
I20261118 03:06:16.217198 25356 webserver.cc:577] Webserver started at http://127.24.137.130:46693/ using document root <none> and password file <none>
I20261118 03:06:16.217370 25356 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:16.217406 25356 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:16.217484 25356 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:16.218297 25356 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/data/instance:
uuid: "c13e18b0db824f0f98438f14e2743947"
format_stamp: "Formatted at 2026-11-18 03:06:16 on dist-test-slave-k60w"
I20261118 03:06:16.218593 25356 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/wal/instance:
uuid: "c13e18b0db824f0f98438f14e2743947"
format_stamp: "Formatted at 2026-11-18 03:06:16 on dist-test-slave-k60w"
I20261118 03:06:16.219583 25356 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:16.220182 25370 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:16.220346 25356 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:16.220403 25356 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/wal
uuid: "c13e18b0db824f0f98438f14e2743947"
format_stamp: "Formatted at 2026-11-18 03:06:16 on dist-test-slave-k60w"
I20261118 03:06:16.220454 25356 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:16.233440 25356 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:16.233731 25356 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:16.233842 25356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:16.234026 25356 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:16.234319 25356 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:16.234349 25356 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:16.234378 25356 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:16.234402 25356 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:16.239769 25356 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.130:41379
I20261118 03:06:16.239820 25483 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.130:41379 every 8 connection(s)
I20261118 03:06:16.240109 25356 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/data/info.pb
I20261118 03:06:16.243974 25484 heartbeater.cc:345] Connected to a master server at 127.24.137.190:42523
I20261118 03:06:16.244088 25484 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:16.244298 25484 heartbeater.cc:508] Master 127.24.137.190:42523 requested a full tablet report, sending...
I20261118 03:06:16.244689 25165 ts_manager.cc:210] Registered new tserver with Master: c13e18b0db824f0f98438f14e2743947 (127.24.137.130:41379)
I20261118 03:06:16.245074 25165 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.130:35795
I20261118 03:06:16.246591 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 25356
I20261118 03:06:16.246657 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/wal/instance
I20261118 03:06:16.247547 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.131:0
--local_ip_for_outbound_sockets=127.24.137.131
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:42523
--builtin_ntp_servers=127.24.137.148:46643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000 with env {}
W20261118 03:06:16.322396 25487 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:16.322549 25487 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:16.322567 25487 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:16.324122 25487 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:16.324179 25487 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.131
I20261118 03:06:16.325784 25487 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:46643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=3000
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:42523
--never_fsync=true
--heap_profile_path=/tmp/kudu.25487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:16.325949 25487 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:16.326087 25487 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:16.328244 25495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:16.328287 25492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:16.328289 25493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:16.328476 25487 server_base.cc:1061] running on GCE node
I20261118 03:06:16.328627 25487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:16.328805 25487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:16.329918 25487 hybrid_clock.cc:648] HybridClock initialized: now 1794971176329905 us; error 32 us; skew 500 ppm
I20261118 03:06:16.330844 25487 webserver.cc:577] Webserver started at http://127.24.137.131:41923/ using document root <none> and password file <none>
I20261118 03:06:16.331012 25487 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:16.331053 25487 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:16.331136 25487 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:16.331912 25487 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/data/instance:
uuid: "d657c9b1c7f54e38a5b52e6291343a99"
format_stamp: "Formatted at 2026-11-18 03:06:16 on dist-test-slave-k60w"
I20261118 03:06:16.332208 25487 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/wal/instance:
uuid: "d657c9b1c7f54e38a5b52e6291343a99"
format_stamp: "Formatted at 2026-11-18 03:06:16 on dist-test-slave-k60w"
I20261118 03:06:16.333196 25487 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:16.333822 25501 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:16.333961 25487 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:16.334024 25487 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/wal
uuid: "d657c9b1c7f54e38a5b52e6291343a99"
format_stamp: "Formatted at 2026-11-18 03:06:16 on dist-test-slave-k60w"
I20261118 03:06:16.334075 25487 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:16.363215 25487 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:16.363472 25487 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:16.363588 25487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:16.363780 25487 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:16.364099 25487 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:16.364130 25487 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:16.364158 25487 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:16.364180 25487 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:16.369550 25487 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.131:42779
I20261118 03:06:16.369623 25614 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.131:42779 every 8 connection(s)
I20261118 03:06:16.369895 25487 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/data/info.pb
I20261118 03:06:16.371563 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 25487
I20261118 03:06:16.371644 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-2/wal/instance
I20261118 03:06:16.374039 25615 heartbeater.cc:345] Connected to a master server at 127.24.137.190:42523
I20261118 03:06:16.374128 25615 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:16.374284 25615 heartbeater.cc:508] Master 127.24.137.190:42523 requested a full tablet report, sending...
I20261118 03:06:16.374562 25165 ts_manager.cc:210] Registered new tserver with Master: d657c9b1c7f54e38a5b52e6291343a99 (127.24.137.131:42779)
I20261118 03:06:16.374954 25165 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.131:44681
I20261118 03:06:16.383687 25126 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20261118 03:06:16.395273 25165 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33892:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20261118 03:06:16.395524 25165 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 03:06:16.401373 25418 tablet_service.cc:1500] Processing CreateTablet for tablet 002a9f5d82e7439d82758a2e8308e00a (DEFAULT_TABLE table=test-workload [id=82efc0a1fac84ef6a7f91c928689ed63]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:16.401736 25418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002a9f5d82e7439d82758a2e8308e00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:16.401827 25549 tablet_service.cc:1500] Processing CreateTablet for tablet 002a9f5d82e7439d82758a2e8308e00a (DEFAULT_TABLE table=test-workload [id=82efc0a1fac84ef6a7f91c928689ed63]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:16.402077 25549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002a9f5d82e7439d82758a2e8308e00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:16.402163 25287 tablet_service.cc:1500] Processing CreateTablet for tablet 002a9f5d82e7439d82758a2e8308e00a (DEFAULT_TABLE table=test-workload [id=82efc0a1fac84ef6a7f91c928689ed63]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:16.402395 25287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002a9f5d82e7439d82758a2e8308e00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:16.404022 25634 tablet_bootstrap.cc:492] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99: Bootstrap starting.
I20261118 03:06:16.404554 25635 tablet_bootstrap.cc:492] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Bootstrap starting.
I20261118 03:06:16.404655 25634 tablet_bootstrap.cc:654] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:16.404906 25634 log.cc:826] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:16.405205 25635 tablet_bootstrap.cc:654] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:16.405448 25635 log.cc:826] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:16.405454 25634 tablet_bootstrap.cc:492] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99: No bootstrap required, opened a new log
I20261118 03:06:16.405505 25634 ts_tablet_manager.cc:1449] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261118 03:06:16.405720 25636 tablet_bootstrap.cc:492] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3: Bootstrap starting.
I20261118 03:06:16.406001 25635 tablet_bootstrap.cc:492] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: No bootstrap required, opened a new log
I20261118 03:06:16.406064 25635 ts_tablet_manager.cc:1449] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20261118 03:06:16.406697 25634 raft_consensus.cc:360] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.406807 25634 raft_consensus.cc:386] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:16.406824 25634 raft_consensus.cc:748] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d657c9b1c7f54e38a5b52e6291343a99, State: Initialized, Role: FOLLOWER
I20261118 03:06:16.406891 25634 consensus_queue.cc:261] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.407117 25634 ts_tablet_manager.cc:1480] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20261118 03:06:16.407307 25635 raft_consensus.cc:360] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.407411 25635 raft_consensus.cc:386] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:16.407428 25635 raft_consensus.cc:748] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c13e18b0db824f0f98438f14e2743947, State: Initialized, Role: FOLLOWER
I20261118 03:06:16.407518 25635 consensus_queue.cc:261] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.407730 25635 ts_tablet_manager.cc:1480] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 03:06:16.407783 25484 heartbeater.cc:500] Master 127.24.137.190:42523 was elected leader, sending a full tablet report...
I20261118 03:06:16.407857 25615 heartbeater.cc:500] Master 127.24.137.190:42523 was elected leader, sending a full tablet report...
I20261118 03:06:16.408597 25636 tablet_bootstrap.cc:654] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:16.408838 25636 log.cc:826] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:16.409370 25636 tablet_bootstrap.cc:492] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3: No bootstrap required, opened a new log
I20261118 03:06:16.409428 25636 ts_tablet_manager.cc:1449] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20261118 03:06:16.410552 25636 raft_consensus.cc:360] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.410662 25636 raft_consensus.cc:386] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:16.410687 25636 raft_consensus.cc:748] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d7feb0bab14f308ec6c222be965cc3, State: Initialized, Role: FOLLOWER
I20261118 03:06:16.410799 25636 consensus_queue.cc:261] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.411031 25636 ts_tablet_manager.cc:1480] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 03:06:16.411041 25353 heartbeater.cc:500] Master 127.24.137.190:42523 was elected leader, sending a full tablet report...
W20261118 03:06:16.490551 25485 tablet.cc:2441] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261118 03:06:16.620375 25616 tablet.cc:2441] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261118 03:06:16.631748 25354 tablet.cc:2441] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 03:06:16.883738 25642 raft_consensus.cc:499] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 03:06:16.883859 25642 raft_consensus.cc:521] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.884187 25642 leader_election.cc:290] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d657c9b1c7f54e38a5b52e6291343a99 (127.24.137.131:42779), c13e18b0db824f0f98438f14e2743947 (127.24.137.130:41379)
I20261118 03:06:16.887094 25438 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "002a9f5d82e7439d82758a2e8308e00a" candidate_uuid: "30d7feb0bab14f308ec6c222be965cc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c13e18b0db824f0f98438f14e2743947" is_pre_election: true
I20261118 03:06:16.887228 25438 raft_consensus.cc:2490] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d7feb0bab14f308ec6c222be965cc3 in term 0.
I20261118 03:06:16.887452 25243 leader_election.cc:304] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d7feb0bab14f308ec6c222be965cc3, c13e18b0db824f0f98438f14e2743947; no voters:
I20261118 03:06:16.887465 25569 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "002a9f5d82e7439d82758a2e8308e00a" candidate_uuid: "30d7feb0bab14f308ec6c222be965cc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d657c9b1c7f54e38a5b52e6291343a99" is_pre_election: true
I20261118 03:06:16.887578 25569 raft_consensus.cc:2490] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d7feb0bab14f308ec6c222be965cc3 in term 0.
I20261118 03:06:16.887583 25642 raft_consensus.cc:2826] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 03:06:16.887621 25642 raft_consensus.cc:499] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 03:06:16.887643 25642 raft_consensus.cc:3096] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:16.888276 25642 raft_consensus.cc:521] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.888411 25642 leader_election.cc:290] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [CANDIDATE]: Term 1 election: Requested vote from peers d657c9b1c7f54e38a5b52e6291343a99 (127.24.137.131:42779), c13e18b0db824f0f98438f14e2743947 (127.24.137.130:41379)
I20261118 03:06:16.888557 25438 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "002a9f5d82e7439d82758a2e8308e00a" candidate_uuid: "30d7feb0bab14f308ec6c222be965cc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c13e18b0db824f0f98438f14e2743947"
I20261118 03:06:16.888638 25438 raft_consensus.cc:3096] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:16.888616 25569 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "002a9f5d82e7439d82758a2e8308e00a" candidate_uuid: "30d7feb0bab14f308ec6c222be965cc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d657c9b1c7f54e38a5b52e6291343a99"
I20261118 03:06:16.888684 25569 raft_consensus.cc:3096] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:16.889371 25569 raft_consensus.cc:2490] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d7feb0bab14f308ec6c222be965cc3 in term 1.
I20261118 03:06:16.889371 25438 raft_consensus.cc:2490] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d7feb0bab14f308ec6c222be965cc3 in term 1.
I20261118 03:06:16.889578 25243 leader_election.cc:304] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d7feb0bab14f308ec6c222be965cc3, c13e18b0db824f0f98438f14e2743947; no voters:
I20261118 03:06:16.889689 25642 raft_consensus.cc:2826] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:16.889846 25642 raft_consensus.cc:705] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 1 LEADER]: Becoming Leader. State: Replica: 30d7feb0bab14f308ec6c222be965cc3, State: Running, Role: LEADER
I20261118 03:06:16.889941 25642 consensus_queue.cc:238] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.890736 25164 catalog_manager.cc:5840] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d7feb0bab14f308ec6c222be965cc3 (127.24.137.129). New cstate: current_term: 1 leader_uuid: "30d7feb0bab14f308ec6c222be965cc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } health_report { overall_health: HEALTHY } } }
I20261118 03:06:16.890847 25645 consensus_queue.cc:1049] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 03:06:16.891000 25645 consensus_queue.cc:1049] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 41379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 03:06:16.991616 25347 generic_service.cc:139] {username='slave'} at 127.0.0.1:46694 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261118 03:06:16.991696 25347 generic_service.cc:175] {username='slave'} at 127.0.0.1:46694 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'false'
I20261118 03:06:16.992067 25347 generic_service.cc:175] {username='slave'} at 127.0.0.1:46694 changed flags via RPC: heartbeat_inject_latency_before_heartbeat_ms from '0' to '2000'
W20261118 03:06:16.992482 25478 generic_service.cc:139] {username='slave'} at 127.0.0.1:39420 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261118 03:06:16.992544 25478 generic_service.cc:175] {username='slave'} at 127.0.0.1:39420 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'false'
I20261118 03:06:16.992825 25478 generic_service.cc:175] {username='slave'} at 127.0.0.1:39420 changed flags via RPC: heartbeat_inject_latency_before_heartbeat_ms from '0' to '5000'
W20261118 03:06:16.993408 25609 generic_service.cc:139] {username='slave'} at 127.0.0.1:44362 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261118 03:06:16.993480 25609 generic_service.cc:175] {username='slave'} at 127.0.0.1:44362 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'false'
I20261118 03:06:16.993855 25609 generic_service.cc:175] {username='slave'} at 127.0.0.1:44362 changed flags via RPC: heartbeat_inject_latency_before_heartbeat_ms from '0' to '5000'
I20261118 03:06:16.994558 25307 consensus_queue.cc:238] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 1.35, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.995043 25569 raft_consensus.cc:1285] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 1 FOLLOWER]: Refusing update from remote peer 30d7feb0bab14f308ec6c222be965cc3: Log matching property violated. Preceding OpId in replica: term: 1 index: 35. Preceding OpId from leader: term: 1 index: 36. (index mismatch)
I20261118 03:06:16.995257 25645 consensus_queue.cc:1049] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20261118 03:06:16.995888 25642 raft_consensus.cc:2983] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 1 LEADER]: Committing config change with OpId 1.36: config changed from index -1 to 36, VOTER c13e18b0db824f0f98438f14e2743947 (127.24.137.130) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } } }
I20261118 03:06:16.996034 25569 raft_consensus.cc:2983] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 1 FOLLOWER]: Committing config change with OpId 1.36: config changed from index -1 to 36, VOTER c13e18b0db824f0f98438f14e2743947 (127.24.137.130) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } } }
W20261118 03:06:16.997251 25347 generic_service.cc:139] {username='slave'} at 127.0.0.1:46694 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261118 03:06:16.997298 25347 generic_service.cc:175] {username='slave'} at 127.0.0.1:46694 changed flags via RPC: follower_reject_update_consensus_requests from 'false' to 'true'
I20261118 03:06:16.997655 25569 tablet_service.cc:2033] Received Run Leader Election RPC: tablet_id: "002a9f5d82e7439d82758a2e8308e00a"
dest_uuid: "d657c9b1c7f54e38a5b52e6291343a99"
from {username='slave'} at 127.0.0.1:44362
I20261118 03:06:16.997722 25569 raft_consensus.cc:499] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20261118 03:06:16.997741 25569 raft_consensus.cc:3096] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 1 FOLLOWER]: Advancing to term 2
I20261118 03:06:16.998198 25569 raft_consensus.cc:521] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:16.998380 25569 leader_election.cc:290] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [CANDIDATE]: Term 2 election: Requested vote from peers 30d7feb0bab14f308ec6c222be965cc3 (127.24.137.129:38305)
I20261118 03:06:17.001116 25307 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "002a9f5d82e7439d82758a2e8308e00a" candidate_uuid: "d657c9b1c7f54e38a5b52e6291343a99" candidate_term: 2 candidate_status { last_received { term: 1 index: 36 } } ignore_live_leader: true dest_uuid: "30d7feb0bab14f308ec6c222be965cc3"
I20261118 03:06:17.001199 25307 raft_consensus.cc:3091] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 1 LEADER]: Stepping down as leader of term 1
I20261118 03:06:17.001214 25307 raft_consensus.cc:748] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 30d7feb0bab14f308ec6c222be965cc3, State: Running, Role: LEADER
I20261118 03:06:17.001251 25307 consensus_queue.cc:261] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 1.36, Last appended by leader: 36, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
I20261118 03:06:17.001317 25307 raft_consensus.cc:3096] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 1 FOLLOWER]: Advancing to term 2
I20261118 03:06:17.001914 25307 raft_consensus.cc:2490] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d657c9b1c7f54e38a5b52e6291343a99 in term 2.
I20261118 03:06:17.002112 25505 leader_election.cc:304] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 30d7feb0bab14f308ec6c222be965cc3, d657c9b1c7f54e38a5b52e6291343a99; no voters:
I20261118 03:06:17.002252 25639 raft_consensus.cc:2826] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 2 FOLLOWER]: Leader election won for term 2
I20261118 03:06:17.002391 25639 raft_consensus.cc:705] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 2 LEADER]: Becoming Leader. State: Replica: d657c9b1c7f54e38a5b52e6291343a99, State: Running, Role: LEADER
I20261118 03:06:17.002460 25639 consensus_queue.cc:238] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 1.36, Last appended by leader: 36, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } }
W20261118 03:06:17.003027 25505 consensus_peers.cc:597] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 -> Peer 30d7feb0bab14f308ec6c222be965cc3 (127.24.137.129:38305): Couldn't send request to peer 30d7feb0bab14f308ec6c222be965cc3. Error code: UNKNOWN_ERROR (1). Status: Illegal state: Rejected: --follower_reject_update_consensus_requests is set to true.. This is attempt 1: this message will repeat every 5th retry.
I20261118 03:06:18.997489 25156 catalog_manager.cc:5840] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 reported cstate change: config changed from index -1 to 36, term changed from 1 to 2, leader changed from 30d7feb0bab14f308ec6c222be965cc3 (127.24.137.129) to <none>, VOTER c13e18b0db824f0f98438f14e2743947 (127.24.137.130) evicted. New cstate: current_term: 2 committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } } }
I20261118 03:06:18.998832 25418 tablet_service.cc:1547] Processing DeleteTablet for tablet 002a9f5d82e7439d82758a2e8308e00a with delete_type TABLET_DATA_TOMBSTONED (TS c13e18b0db824f0f98438f14e2743947 not found in new config with opid_index 36) from {username='slave'} at 127.0.0.1:39408
I20261118 03:06:18.999068 25678 tablet_replica.cc:333] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: stopping tablet replica
I20261118 03:06:18.999152 25678 raft_consensus.cc:2261] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 03:06:18.999217 25678 raft_consensus.cc:2290] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 03:06:18.999501 25678 ts_tablet_manager.cc:1962] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20261118 03:06:19.000769 25678 ts_tablet_manager.cc:1975] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.35
I20261118 03:06:19.000824 25678 log.cc:1199] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Deleting WAL directory at /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestKudu2147.1794971175865063-25126-0/minicluster-data/ts-1/wal/wals/002a9f5d82e7439d82758a2e8308e00a
I20261118 03:06:19.001206 25152 catalog_manager.cc:5065] TS c13e18b0db824f0f98438f14e2743947 (127.24.137.130:41379): tablet 002a9f5d82e7439d82758a2e8308e00a (table test-workload [id=82efc0a1fac84ef6a7f91c928689ed63]) successfully deleted
W20261118 03:06:19.029722 25347 generic_service.cc:139] {username='slave'} at 127.0.0.1:46694 forcing change of non-runtime-safe flag follower_reject_update_consensus_requests
I20261118 03:06:19.029804 25347 generic_service.cc:175] {username='slave'} at 127.0.0.1:46694 changed flags via RPC: follower_reject_update_consensus_requests from 'true' to 'false'
I20261118 03:06:19.030889 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 1.36 <uninitialized op> 2.37
I20261118 03:06:19.132230 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 1.36 <uninitialized op> 2.37
I20261118 03:06:19.333606 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 1.36 <uninitialized op> 2.37
I20261118 03:06:19.634907 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 1.36 <uninitialized op> 2.37
I20261118 03:06:20.036268 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 2.37 <uninitialized op> 2.37
I20261118 03:06:20.537606 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 2.37 <uninitialized op> 2.37
I20261118 03:06:20.999408 25156 catalog_manager.cc:5840] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 reported cstate change: leader changed from <none> to d657c9b1c7f54e38a5b52e6291343a99 (127.24.137.131). New cstate: current_term: 2 leader_uuid: "d657c9b1c7f54e38a5b52e6291343a99" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } } }
I20261118 03:06:21.138836 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 2.37 <uninitialized op> 2.37
I20261118 03:06:21.840193 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 2.37 <uninitialized op> 2.37
I20261118 03:06:21.998104 25569 consensus_queue.cc:238] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 2.37, Last appended by leader: 36, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 41379 } attrs { promote: true } }
I20261118 03:06:21.998963 25307 raft_consensus.cc:1285] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 2 FOLLOWER]: Refusing update from remote peer d657c9b1c7f54e38a5b52e6291343a99: Log matching property violated. Preceding OpId in replica: term: 2 index: 37. Preceding OpId from leader: term: 2 index: 38. (index mismatch)
I20261118 03:06:21.999234 25692 consensus_queue.cc:1049] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20261118 03:06:22.000026 25691 raft_consensus.cc:2983] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 [term 2 LEADER]: Committing config change with OpId 2.38: config changed from index 36 to 38, NON_VOTER c13e18b0db824f0f98438f14e2743947 (127.24.137.130) added. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 41379 } attrs { promote: true } } }
I20261118 03:06:22.000206 25307 raft_consensus.cc:2983] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 [term 2 FOLLOWER]: Committing config change with OpId 2.38: config changed from index 36 to 38, NON_VOTER c13e18b0db824f0f98438f14e2743947 (127.24.137.130) added. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 41379 } attrs { promote: true } } }
I20261118 03:06:22.001070 25150 catalog_manager.cc:5247] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 002a9f5d82e7439d82758a2e8308e00a with cas_config_opid_index 36: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20261118 03:06:22.001757 25505 consensus_peers.cc:597] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99 -> Peer c13e18b0db824f0f98438f14e2743947 (127.24.137.130:41379): Couldn't send request to peer c13e18b0db824f0f98438f14e2743947. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20261118 03:06:22.641428 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 2.38 <uninitialized op> 2.38
I20261118 03:06:23.001202 25156 catalog_manager.cc:5840] T 002a9f5d82e7439d82758a2e8308e00a P 30d7feb0bab14f308ec6c222be965cc3 reported cstate change: config changed from index 36 to 38, NON_VOTER c13e18b0db824f0f98438f14e2743947 (127.24.137.130) added. New cstate: current_term: 2 leader_uuid: "d657c9b1c7f54e38a5b52e6291343a99" committed_config { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 41379 } attrs { promote: true } } }
I20261118 03:06:23.542706 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 2.38 <uninitialized op> 2.38
I20261118 03:06:24.543928 25703 ts_tablet_manager.cc:979] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Initiating tablet copy from peer d657c9b1c7f54e38a5b52e6291343a99 (127.24.137.131:42779)
I20261118 03:06:24.544142 25126 cluster_itest_util.cc:258] Not converged past 1 yet: 2.38 <uninitialized op> 2.38
I20261118 03:06:24.544167 25703 tablet_copy_client.cc:323] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: tablet copy: Beginning tablet copy session from remote peer at address 127.24.137.131:42779
I20261118 03:06:24.546617 25589 tablet_copy_service.cc:140] P d657c9b1c7f54e38a5b52e6291343a99: Received BeginTabletCopySession request for tablet 002a9f5d82e7439d82758a2e8308e00a from peer c13e18b0db824f0f98438f14e2743947 ({username='slave'} at 127.24.137.130:48251)
I20261118 03:06:24.546693 25589 tablet_copy_service.cc:161] P d657c9b1c7f54e38a5b52e6291343a99: Beginning new tablet copy session on tablet 002a9f5d82e7439d82758a2e8308e00a from peer c13e18b0db824f0f98438f14e2743947 at {username='slave'} at 127.24.137.130:48251: session id = c13e18b0db824f0f98438f14e2743947-002a9f5d82e7439d82758a2e8308e00a
I20261118 03:06:24.547222 25589 tablet_copy_source_session.cc:215] T 002a9f5d82e7439d82758a2e8308e00a P d657c9b1c7f54e38a5b52e6291343a99: Tablet Copy: opened 0 blocks and 1 log segments
I20261118 03:06:24.547680 25703 ts_tablet_manager.cc:1962] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Deleting tablet data with delete state TABLET_DATA_COPYING
I20261118 03:06:24.549099 25703 ts_tablet_manager.cc:1975] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.35
I20261118 03:06:24.549203 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002a9f5d82e7439d82758a2e8308e00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:24.549808 25703 tablet_copy_client.cc:806] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: tablet copy: Starting download of 0 data blocks...
I20261118 03:06:24.549903 25703 tablet_copy_client.cc:670] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: tablet copy: Starting download of 1 WAL segments...
I20261118 03:06:24.550868 25703 tablet_copy_client.cc:538] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261118 03:06:24.551424 25703 tablet_bootstrap.cc:492] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Bootstrap starting.
I20261118 03:06:24.557801 25703 tablet_bootstrap.cc:492] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Bootstrap replayed 1/1 log segments. Stats: ops{read=38 overwritten=0 applied=38 ignored=0} inserts{seen=1700 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 03:06:24.558022 25703 tablet_bootstrap.cc:492] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Bootstrap complete.
I20261118 03:06:24.558130 25703 ts_tablet_manager.cc:1449] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20261118 03:06:24.558280 25703 raft_consensus.cc:360] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 41379 } attrs { promote: true } }
I20261118 03:06:24.558342 25703 raft_consensus.cc:748] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c13e18b0db824f0f98438f14e2743947, State: Initialized, Role: LEARNER
I20261118 03:06:24.558373 25703 consensus_queue.cc:261] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 38, Last appended: 2.38, Last appended by leader: 38, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "d657c9b1c7f54e38a5b52e6291343a99" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 42779 } } peers { permanent_uuid: "30d7feb0bab14f308ec6c222be965cc3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 38305 } } peers { permanent_uuid: "c13e18b0db824f0f98438f14e2743947" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 41379 } attrs { promote: true } }
I20261118 03:06:24.558540 25703 ts_tablet_manager.cc:1480] T 002a9f5d82e7439d82758a2e8308e00a P c13e18b0db824f0f98438f14e2743947: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:24.558856 25589 tablet_copy_service.cc:342] P d657c9b1c7f54e38a5b52e6291343a99: Request end of tablet copy session c13e18b0db824f0f98438f14e2743947-002a9f5d82e7439d82758a2e8308e00a received from {username='slave'} at 127.24.137.130:48251
I20261118 03:06:24.558921 25589 tablet_copy_service.cc:434] P d657c9b1c7f54e38a5b52e6291343a99: ending tablet copy session c13e18b0db824f0f98438f14e2743947-002a9f5d82e7439d82758a2e8308e00a on tablet 002a9f5d82e7439d82758a2e8308e00a with peer c13e18b0db824f0f98438f14e2743947
I20261118 03:06:25.545629 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 25225
I20261118 03:06:25.550096 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 25356
I20261118 03:06:25.554105 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 25487
I20261118 03:06:25.558033 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 25134
2026-11-18T03:06:25Z chronyd exiting
[ OK ] RaftConfigChangeITest.TestKudu2147 (9706 ms)
[ RUN ] RaftConfigChangeITest.TestNonVoterPromotion
2026-11-18T03:06:25Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-18T03:06:25Z Disabled control of system clock
I20261118 03:06:25.582974 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:33893
--webserver_interface=127.24.137.190
--webserver_port=0
--builtin_ntp_servers=127.24.137.148:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.137.190:33893
--raft_prepare_replacement_before_eviction=true
--catalog_manager_evict_excess_replicas=false with env {}
W20261118 03:06:25.657168 25712 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:25.657348 25712 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:25.657374 25712 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:25.658962 25712 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 03:06:25.659014 25712 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:25.659031 25712 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 03:06:25.659045 25712 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 03:06:25.660665 25712 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/wal
--catalog_manager_evict_excess_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.24.137.190:33893
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:33893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.137.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.25712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:25.660871 25712 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:25.661028 25712 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20261118 03:06:25.663424 25712 server_base.cc:1061] running on GCE node
W20261118 03:06:25.663527 25717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:25.663532 25718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:25.663753 25720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:25.663956 25712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:25.664173 25712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:25.665297 25712 hybrid_clock.cc:648] HybridClock initialized: now 1794971185665285 us; error 30 us; skew 500 ppm
I20261118 03:06:25.666468 25712 webserver.cc:577] Webserver started at http://127.24.137.190:43871/ using document root <none> and password file <none>
I20261118 03:06:25.666666 25712 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:25.666705 25712 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:25.666797 25712 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:25.667619 25712 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/data/instance:
uuid: "0b014f550d934e29ac628301e5afddb0"
format_stamp: "Formatted at 2026-11-18 03:06:25 on dist-test-slave-k60w"
I20261118 03:06:25.667923 25712 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/wal/instance:
uuid: "0b014f550d934e29ac628301e5afddb0"
format_stamp: "Formatted at 2026-11-18 03:06:25 on dist-test-slave-k60w"
I20261118 03:06:25.669018 25712 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261118 03:06:25.669705 25726 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:25.669842 25712 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:25.669898 25712 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/wal
uuid: "0b014f550d934e29ac628301e5afddb0"
format_stamp: "Formatted at 2026-11-18 03:06:25 on dist-test-slave-k60w"
I20261118 03:06:25.669953 25712 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:25.687212 25712 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:25.687466 25712 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:25.687568 25712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:25.691104 25712 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.190:33893
I20261118 03:06:25.691134 25778 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.190:33893 every 8 connection(s)
I20261118 03:06:25.691449 25712 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/data/info.pb
I20261118 03:06:25.691999 25779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:25.694195 25779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0: Bootstrap starting.
I20261118 03:06:25.694766 25779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:25.694970 25779 log.cc:826] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:25.695523 25779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0: No bootstrap required, opened a new log
I20261118 03:06:25.696369 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 25712
I20261118 03:06:25.696460 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/master-0/wal/instance
I20261118 03:06:25.696903 25779 raft_consensus.cc:360] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b014f550d934e29ac628301e5afddb0" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 33893 } }
I20261118 03:06:25.697088 25779 raft_consensus.cc:386] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:25.697154 25779 raft_consensus.cc:748] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b014f550d934e29ac628301e5afddb0, State: Initialized, Role: FOLLOWER
I20261118 03:06:25.697337 25779 consensus_queue.cc:261] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b014f550d934e29ac628301e5afddb0" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 33893 } }
I20261118 03:06:25.697403 25779 raft_consensus.cc:400] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261118 03:06:25.697432 25779 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261118 03:06:25.697475 25779 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:25.698148 25779 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b014f550d934e29ac628301e5afddb0" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 33893 } }
I20261118 03:06:25.698253 25779 leader_election.cc:304] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b014f550d934e29ac628301e5afddb0; no voters:
I20261118 03:06:25.698402 25779 leader_election.cc:290] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261118 03:06:25.698451 25784 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:25.698590 25784 raft_consensus.cc:705] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [term 1 LEADER]: Becoming Leader. State: Replica: 0b014f550d934e29ac628301e5afddb0, State: Running, Role: LEADER
I20261118 03:06:25.698719 25784 consensus_queue.cc:238] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b014f550d934e29ac628301e5afddb0" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 33893 } }
I20261118 03:06:25.698719 25779 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 03:06:25.700320 25785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b014f550d934e29ac628301e5afddb0. Latest consensus state: current_term: 1 leader_uuid: "0b014f550d934e29ac628301e5afddb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b014f550d934e29ac628301e5afddb0" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 33893 } } }
I20261118 03:06:25.700537 25785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [sys.catalog]: This master's current role is: LEADER
W20261118 03:06:25.700788 25798 catalog_manager.cc:1609] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20261118 03:06:25.700832 25798 catalog_manager.cc:898] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20261118 03:06:25.700882 25802 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 03:06:25.700897 25786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b014f550d934e29ac628301e5afddb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b014f550d934e29ac628301e5afddb0" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 33893 } } }
I20261118 03:06:25.700997 25786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0 [sys.catalog]: This master's current role is: LEADER
I20261118 03:06:25.701077 25802 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 03:06:25.702369 25802 catalog_manager.cc:1398] Generated new cluster ID: f3169dc3b3f2484b84a6722416e61f38
I20261118 03:06:25.702421 25802 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 03:06:25.734683 25802 catalog_manager.cc:1421] Generated new certificate authority record
I20261118 03:06:25.735213 25802 catalog_manager.cc:1555] Loading token signing keys...
I20261118 03:06:25.742893 25802 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 0b014f550d934e29ac628301e5afddb0: Generated new TSK 0
I20261118 03:06:25.743096 25802 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 03:06:25.752287 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.129:0
--local_ip_for_outbound_sockets=127.24.137.129
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:33893
--builtin_ntp_servers=127.24.137.148:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20261118 03:06:25.827796 25804 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:25.827948 25804 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:25.827967 25804 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:25.829619 25804 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:25.829682 25804 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.129
I20261118 03:06:25.831324 25804 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:33893
--never_fsync=true
--heap_profile_path=/tmp/kudu.25804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:25.831532 25804 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:25.831713 25804 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:25.833982 25809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:25.834118 25804 server_base.cc:1061] running on GCE node
W20261118 03:06:25.834016 25812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:25.834029 25810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:25.834355 25804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:25.834533 25804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:25.835654 25804 hybrid_clock.cc:648] HybridClock initialized: now 1794971185835641 us; error 29 us; skew 500 ppm
I20261118 03:06:25.836602 25804 webserver.cc:577] Webserver started at http://127.24.137.129:36779/ using document root <none> and password file <none>
I20261118 03:06:25.836779 25804 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:25.836818 25804 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:25.836905 25804 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:25.837755 25804 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/data/instance:
uuid: "ed230c02f7824013bf487c770d266ad3"
format_stamp: "Formatted at 2026-11-18 03:06:25 on dist-test-slave-k60w"
I20261118 03:06:25.838060 25804 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/wal/instance:
uuid: "ed230c02f7824013bf487c770d266ad3"
format_stamp: "Formatted at 2026-11-18 03:06:25 on dist-test-slave-k60w"
I20261118 03:06:25.839200 25804 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:25.839859 25818 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:25.840008 25804 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:25.840068 25804 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/wal
uuid: "ed230c02f7824013bf487c770d266ad3"
format_stamp: "Formatted at 2026-11-18 03:06:25 on dist-test-slave-k60w"
I20261118 03:06:25.840122 25804 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:25.870723 25804 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:25.871002 25804 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:25.871117 25804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:25.871313 25804 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:25.871624 25804 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:25.871655 25804 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:25.871685 25804 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:25.871702 25804 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:25.877301 25804 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.129:34337
I20261118 03:06:25.877352 25931 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.129:34337 every 8 connection(s)
I20261118 03:06:25.877676 25804 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/data/info.pb
I20261118 03:06:25.881767 25932 heartbeater.cc:345] Connected to a master server at 127.24.137.190:33893
I20261118 03:06:25.881865 25932 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:25.882040 25932 heartbeater.cc:508] Master 127.24.137.190:33893 requested a full tablet report, sending...
I20261118 03:06:25.882511 25743 ts_manager.cc:210] Registered new tserver with Master: ed230c02f7824013bf487c770d266ad3 (127.24.137.129:34337)
I20261118 03:06:25.883154 25743 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.129:38369
I20261118 03:06:25.887035 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 25804
I20261118 03:06:25.887104 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-0/wal/instance
I20261118 03:06:25.888079 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.130:0
--local_ip_for_outbound_sockets=127.24.137.130
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:33893
--builtin_ntp_servers=127.24.137.148:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20261118 03:06:25.962450 25935 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:25.962607 25935 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:25.962628 25935 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:25.964251 25935 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:25.964320 25935 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.130
I20261118 03:06:25.966012 25935 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:33893
--never_fsync=true
--heap_profile_path=/tmp/kudu.25935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:25.966219 25935 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:25.966385 25935 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:25.968701 25941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:25.968729 25940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:25.968866 25935 server_base.cc:1061] running on GCE node
W20261118 03:06:25.968652 25943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:25.969036 25935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:25.969201 25935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:25.970336 25935 hybrid_clock.cc:648] HybridClock initialized: now 1794971185970320 us; error 38 us; skew 500 ppm
I20261118 03:06:25.971274 25935 webserver.cc:577] Webserver started at http://127.24.137.130:32785/ using document root <none> and password file <none>
I20261118 03:06:25.971469 25935 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:25.971509 25935 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:25.971601 25935 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:25.972427 25935 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/data/instance:
uuid: "b3216cd0da36498b85fefc2a5f0cbe7e"
format_stamp: "Formatted at 2026-11-18 03:06:25 on dist-test-slave-k60w"
I20261118 03:06:25.972738 25935 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/wal/instance:
uuid: "b3216cd0da36498b85fefc2a5f0cbe7e"
format_stamp: "Formatted at 2026-11-18 03:06:25 on dist-test-slave-k60w"
I20261118 03:06:25.973764 25935 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261118 03:06:25.974360 25949 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:25.974510 25935 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:25.974566 25935 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/wal
uuid: "b3216cd0da36498b85fefc2a5f0cbe7e"
format_stamp: "Formatted at 2026-11-18 03:06:25 on dist-test-slave-k60w"
I20261118 03:06:25.974619 25935 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:25.985070 25935 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:25.985335 25935 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:25.985443 25935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:25.985666 25935 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:25.985962 25935 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:25.985992 25935 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:25.986021 25935 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:25.986044 25935 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:25.991433 25935 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.130:46425
I20261118 03:06:25.991493 26062 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.130:46425 every 8 connection(s)
I20261118 03:06:25.991779 25935 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/data/info.pb
I20261118 03:06:25.992012 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 25935
I20261118 03:06:25.992090 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-1/wal/instance
I20261118 03:06:25.993279 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.131:0
--local_ip_for_outbound_sockets=127.24.137.131
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:33893
--builtin_ntp_servers=127.24.137.148:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20261118 03:06:25.996029 26063 heartbeater.cc:345] Connected to a master server at 127.24.137.190:33893
I20261118 03:06:25.996145 26063 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:25.996335 26063 heartbeater.cc:508] Master 127.24.137.190:33893 requested a full tablet report, sending...
I20261118 03:06:25.996703 25743 ts_manager.cc:210] Registered new tserver with Master: b3216cd0da36498b85fefc2a5f0cbe7e (127.24.137.130:46425)
I20261118 03:06:25.997093 25743 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.130:51351
W20261118 03:06:26.069608 26066 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:26.069756 26066 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:26.069782 26066 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:26.071309 26066 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:26.071369 26066 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.131
I20261118 03:06:26.072916 26066 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:33893
--never_fsync=true
--heap_profile_path=/tmp/kudu.26066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:26.073110 26066 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:26.073263 26066 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:26.075488 26071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:26.075516 26074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:26.075539 26066 server_base.cc:1061] running on GCE node
W20261118 03:06:26.075732 26072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:26.075881 26066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:26.076062 26066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:26.077167 26066 hybrid_clock.cc:648] HybridClock initialized: now 1794971186077157 us; error 26 us; skew 500 ppm
I20261118 03:06:26.078177 26066 webserver.cc:577] Webserver started at http://127.24.137.131:46783/ using document root <none> and password file <none>
I20261118 03:06:26.078387 26066 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:26.078428 26066 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:26.078516 26066 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:26.079282 26066 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/data/instance:
uuid: "44c9837255df4c0da68aa6475c9458d8"
format_stamp: "Formatted at 2026-11-18 03:06:26 on dist-test-slave-k60w"
I20261118 03:06:26.079586 26066 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/wal/instance:
uuid: "44c9837255df4c0da68aa6475c9458d8"
format_stamp: "Formatted at 2026-11-18 03:06:26 on dist-test-slave-k60w"
I20261118 03:06:26.080596 26066 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:26.081156 26080 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:26.081318 26066 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:26.081375 26066 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/wal
uuid: "44c9837255df4c0da68aa6475c9458d8"
format_stamp: "Formatted at 2026-11-18 03:06:26 on dist-test-slave-k60w"
I20261118 03:06:26.081430 26066 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:26.097210 26066 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:26.097484 26066 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:26.097621 26066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:26.097801 26066 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:26.098106 26066 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:26.098137 26066 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:26.098166 26066 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:26.098183 26066 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:26.103483 26066 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.131:41879
I20261118 03:06:26.103529 26193 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.131:41879 every 8 connection(s)
I20261118 03:06:26.103816 26066 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/data/info.pb
I20261118 03:06:26.107654 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 26066
I20261118 03:06:26.107743 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-2/wal/instance
I20261118 03:06:26.108533 26194 heartbeater.cc:345] Connected to a master server at 127.24.137.190:33893
I20261118 03:06:26.108634 26194 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:26.108793 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.132:0
--local_ip_for_outbound_sockets=127.24.137.132
--webserver_interface=127.24.137.132
--webserver_port=0
--tserver_master_addrs=127.24.137.190:33893
--builtin_ntp_servers=127.24.137.148:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20261118 03:06:26.108810 26194 heartbeater.cc:508] Master 127.24.137.190:33893 requested a full tablet report, sending...
I20261118 03:06:26.109174 25743 ts_manager.cc:210] Registered new tserver with Master: 44c9837255df4c0da68aa6475c9458d8 (127.24.137.131:41879)
I20261118 03:06:26.109525 25743 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.131:58915
W20261118 03:06:26.183601 26197 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:26.183756 26197 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:26.183782 26197 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:26.185410 26197 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:26.185474 26197 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.132
I20261118 03:06:26.187084 26197 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.24.137.132
--webserver_port=0
--tserver_master_addrs=127.24.137.190:33893
--never_fsync=true
--heap_profile_path=/tmp/kudu.26197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:26.187281 26197 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:26.187446 26197 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:26.189756 26205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:26.189775 26202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:26.189919 26197 server_base.cc:1061] running on GCE node
W20261118 03:06:26.189759 26203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:26.190100 26197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:26.190308 26197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:26.191437 26197 hybrid_clock.cc:648] HybridClock initialized: now 1794971186191423 us; error 29 us; skew 500 ppm
I20261118 03:06:26.192420 26197 webserver.cc:577] Webserver started at http://127.24.137.132:34611/ using document root <none> and password file <none>
I20261118 03:06:26.192622 26197 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:26.192669 26197 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:26.192764 26197 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:26.193573 26197 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/data/instance:
uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb"
format_stamp: "Formatted at 2026-11-18 03:06:26 on dist-test-slave-k60w"
I20261118 03:06:26.193893 26197 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/wal/instance:
uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb"
format_stamp: "Formatted at 2026-11-18 03:06:26 on dist-test-slave-k60w"
I20261118 03:06:26.194912 26197 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20261118 03:06:26.195521 26211 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:26.195652 26197 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:26.195709 26197 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/wal
uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb"
format_stamp: "Formatted at 2026-11-18 03:06:26 on dist-test-slave-k60w"
I20261118 03:06:26.195762 26197 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:26.218438 26197 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:26.218706 26197 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:26.218820 26197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:26.219020 26197 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:26.219357 26197 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:26.219389 26197 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:26.219419 26197 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:26.219435 26197 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:26.224866 26197 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.132:43763
I20261118 03:06:26.224912 26324 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.132:43763 every 8 connection(s)
I20261118 03:06:26.225195 26197 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/data/info.pb
I20261118 03:06:26.229276 26325 heartbeater.cc:345] Connected to a master server at 127.24.137.190:33893
I20261118 03:06:26.229374 26325 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:26.229597 26325 heartbeater.cc:508] Master 127.24.137.190:33893 requested a full tablet report, sending...
I20261118 03:06:26.229943 25743 ts_manager.cc:210] Registered new tserver with Master: cc92bc45b19d4fc0a95f6cf24766eaeb (127.24.137.132:43763)
I20261118 03:06:26.230297 25743 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.132:35651
I20261118 03:06:26.233258 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 26197
I20261118 03:06:26.233328 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestNonVoterPromotion.1794971175865063-25126-0/minicluster-data/ts-3/wal/instance
I20261118 03:06:26.234328 25126 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261118 03:06:26.244692 25743 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43884:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 03:06:26.250509 26128 tablet_service.cc:1500] Processing CreateTablet for tablet 0a463bcede924fc883b321026eaf2017 (DEFAULT_TABLE table=test-workload [id=26001fc66474427eab73ad0a274ff66c]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:26.250777 26128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a463bcede924fc883b321026eaf2017. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:26.251435 26259 tablet_service.cc:1500] Processing CreateTablet for tablet 0a463bcede924fc883b321026eaf2017 (DEFAULT_TABLE table=test-workload [id=26001fc66474427eab73ad0a274ff66c]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:26.251677 26259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a463bcede924fc883b321026eaf2017. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:26.251744 25997 tablet_service.cc:1500] Processing CreateTablet for tablet 0a463bcede924fc883b321026eaf2017 (DEFAULT_TABLE table=test-workload [id=26001fc66474427eab73ad0a274ff66c]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:26.251945 25997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a463bcede924fc883b321026eaf2017. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:26.253166 26347 tablet_bootstrap.cc:492] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8: Bootstrap starting.
I20261118 03:06:26.253726 26347 tablet_bootstrap.cc:654] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:26.253962 26347 log.cc:826] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:26.254199 26349 tablet_bootstrap.cc:492] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e: Bootstrap starting.
I20261118 03:06:26.254331 26348 tablet_bootstrap.cc:492] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb: Bootstrap starting.
I20261118 03:06:26.254688 26347 tablet_bootstrap.cc:492] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8: No bootstrap required, opened a new log
I20261118 03:06:26.254772 26347 ts_tablet_manager.cc:1449] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261118 03:06:26.254808 26349 tablet_bootstrap.cc:654] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:26.254850 26348 tablet_bootstrap.cc:654] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:26.255074 26348 log.cc:826] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:26.255115 26349 log.cc:826] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:26.255558 26348 tablet_bootstrap.cc:492] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb: No bootstrap required, opened a new log
I20261118 03:06:26.255606 26348 ts_tablet_manager.cc:1449] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:26.256068 26347 raft_consensus.cc:360] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } }
I20261118 03:06:26.256174 26347 raft_consensus.cc:386] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:26.256199 26347 raft_consensus.cc:748] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c9837255df4c0da68aa6475c9458d8, State: Initialized, Role: FOLLOWER
I20261118 03:06:26.256281 26347 consensus_queue.cc:261] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } }
I20261118 03:06:26.256505 26347 ts_tablet_manager.cc:1480] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 03:06:26.256521 26194 heartbeater.cc:500] Master 127.24.137.190:33893 was elected leader, sending a full tablet report...
I20261118 03:06:26.256897 26348 raft_consensus.cc:360] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } }
I20261118 03:06:26.257035 26348 raft_consensus.cc:386] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:26.257058 26348 raft_consensus.cc:748] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc92bc45b19d4fc0a95f6cf24766eaeb, State: Initialized, Role: FOLLOWER
I20261118 03:06:26.257139 26348 consensus_queue.cc:261] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } }
I20261118 03:06:26.257351 26348 ts_tablet_manager.cc:1480] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 03:06:26.257390 26325 heartbeater.cc:500] Master 127.24.137.190:33893 was elected leader, sending a full tablet report...
I20261118 03:06:26.257879 26349 tablet_bootstrap.cc:492] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e: No bootstrap required, opened a new log
I20261118 03:06:26.257952 26349 ts_tablet_manager.cc:1449] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20261118 03:06:26.259052 26349 raft_consensus.cc:360] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } }
I20261118 03:06:26.259155 26349 raft_consensus.cc:386] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:26.259179 26349 raft_consensus.cc:748] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3216cd0da36498b85fefc2a5f0cbe7e, State: Initialized, Role: FOLLOWER
I20261118 03:06:26.259254 26349 consensus_queue.cc:261] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } }
I20261118 03:06:26.259449 26349 ts_tablet_manager.cc:1480] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:26.259490 26063 heartbeater.cc:500] Master 127.24.137.190:33893 was elected leader, sending a full tablet report...
W20261118 03:06:26.354318 26195 tablet.cc:2441] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 03:06:26.449878 26354 raft_consensus.cc:499] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 03:06:26.449998 26354 raft_consensus.cc:521] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } }
I20261118 03:06:26.450342 26354 leader_election.cc:290] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44c9837255df4c0da68aa6475c9458d8 (127.24.137.131:41879), b3216cd0da36498b85fefc2a5f0cbe7e (127.24.137.130:46425)
I20261118 03:06:26.452998 26148 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0a463bcede924fc883b321026eaf2017" candidate_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44c9837255df4c0da68aa6475c9458d8" is_pre_election: true
I20261118 03:06:26.453161 26148 raft_consensus.cc:2490] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc92bc45b19d4fc0a95f6cf24766eaeb in term 0.
I20261118 03:06:26.453428 26215 leader_election.cc:304] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c9837255df4c0da68aa6475c9458d8, cc92bc45b19d4fc0a95f6cf24766eaeb; no voters:
I20261118 03:06:26.453428 26017 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0a463bcede924fc883b321026eaf2017" candidate_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" is_pre_election: true
I20261118 03:06:26.453572 26017 raft_consensus.cc:2490] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc92bc45b19d4fc0a95f6cf24766eaeb in term 0.
I20261118 03:06:26.453581 26354 raft_consensus.cc:2826] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 03:06:26.453636 26354 raft_consensus.cc:499] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 03:06:26.453744 26354 raft_consensus.cc:3096] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:26.454337 26354 raft_consensus.cc:521] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } }
I20261118 03:06:26.454452 26354 leader_election.cc:290] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [CANDIDATE]: Term 1 election: Requested vote from peers 44c9837255df4c0da68aa6475c9458d8 (127.24.137.131:41879), b3216cd0da36498b85fefc2a5f0cbe7e (127.24.137.130:46425)
I20261118 03:06:26.454609 26148 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0a463bcede924fc883b321026eaf2017" candidate_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44c9837255df4c0da68aa6475c9458d8"
I20261118 03:06:26.454674 26148 raft_consensus.cc:3096] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:26.454643 26017 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0a463bcede924fc883b321026eaf2017" candidate_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e"
I20261118 03:06:26.454713 26017 raft_consensus.cc:3096] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:26.455510 26148 raft_consensus.cc:2490] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc92bc45b19d4fc0a95f6cf24766eaeb in term 1.
I20261118 03:06:26.455546 26017 raft_consensus.cc:2490] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc92bc45b19d4fc0a95f6cf24766eaeb in term 1.
I20261118 03:06:26.455663 26215 leader_election.cc:304] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c9837255df4c0da68aa6475c9458d8, cc92bc45b19d4fc0a95f6cf24766eaeb; no voters:
I20261118 03:06:26.455757 26354 raft_consensus.cc:2826] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:26.455901 26354 raft_consensus.cc:705] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 1 LEADER]: Becoming Leader. State: Replica: cc92bc45b19d4fc0a95f6cf24766eaeb, State: Running, Role: LEADER
I20261118 03:06:26.455977 26354 consensus_queue.cc:238] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } }
I20261118 03:06:26.456894 26359 consensus_queue.cc:1049] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 03:06:26.456979 26354 consensus_queue.cc:1049] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 03:06:26.457027 25743 catalog_manager.cc:5840] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb reported cstate change: term changed from 0 to 1, leader changed from <none> to cc92bc45b19d4fc0a95f6cf24766eaeb (127.24.137.132). New cstate: current_term: 1 leader_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } health_report { overall_health: HEALTHY } } }
I20261118 03:06:26.483824 26279 consensus_queue.cc:238] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: true } }
I20261118 03:06:26.484551 26017 raft_consensus.cc:1285] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [term 1 FOLLOWER]: Refusing update from remote peer cc92bc45b19d4fc0a95f6cf24766eaeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20261118 03:06:26.484546 26148 raft_consensus.cc:1285] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [term 1 FOLLOWER]: Refusing update from remote peer cc92bc45b19d4fc0a95f6cf24766eaeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20261118 03:06:26.484786 26357 consensus_queue.cc:1049] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20261118 03:06:26.484926 26357 consensus_queue.cc:1049] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20261118 03:06:26.485453 26357 raft_consensus.cc:2983] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ed230c02f7824013bf487c770d266ad3 (127.24.137.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: true } } }
I20261118 03:06:26.485612 26148 raft_consensus.cc:2983] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ed230c02f7824013bf487c770d266ad3 (127.24.137.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: true } } }
I20261118 03:06:26.485817 26017 raft_consensus.cc:2983] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ed230c02f7824013bf487c770d266ad3 (127.24.137.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: true } } }
I20261118 03:06:26.486990 25743 catalog_manager.cc:5840] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb reported cstate change: config changed from index -1 to 2, NON_VOTER ed230c02f7824013bf487c770d266ad3 (127.24.137.129) added. New cstate: current_term: 1 leader_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20261118 03:06:26.488170 26215 consensus_peers.cc:597] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb -> Peer ed230c02f7824013bf487c770d266ad3 (127.24.137.129:34337): Couldn't send request to peer ed230c02f7824013bf487c770d266ad3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a463bcede924fc883b321026eaf2017. This is attempt 1: this message will repeat every 5th retry.
I20261118 03:06:26.883842 25932 heartbeater.cc:500] Master 127.24.137.190:33893 was elected leader, sending a full tablet report...
I20261118 03:06:26.951243 26367 ts_tablet_manager.cc:979] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: Initiating tablet copy from peer cc92bc45b19d4fc0a95f6cf24766eaeb (127.24.137.132:43763)
I20261118 03:06:26.951511 26367 tablet_copy_client.cc:323] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: tablet copy: Beginning tablet copy session from remote peer at address 127.24.137.132:43763
I20261118 03:06:26.954332 26299 tablet_copy_service.cc:140] P cc92bc45b19d4fc0a95f6cf24766eaeb: Received BeginTabletCopySession request for tablet 0a463bcede924fc883b321026eaf2017 from peer ed230c02f7824013bf487c770d266ad3 ({username='slave'} at 127.24.137.129:38725)
I20261118 03:06:26.954411 26299 tablet_copy_service.cc:161] P cc92bc45b19d4fc0a95f6cf24766eaeb: Beginning new tablet copy session on tablet 0a463bcede924fc883b321026eaf2017 from peer ed230c02f7824013bf487c770d266ad3 at {username='slave'} at 127.24.137.129:38725: session id = ed230c02f7824013bf487c770d266ad3-0a463bcede924fc883b321026eaf2017
I20261118 03:06:26.954937 26299 tablet_copy_source_session.cc:215] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb: Tablet Copy: opened 0 blocks and 1 log segments
I20261118 03:06:26.955667 26367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a463bcede924fc883b321026eaf2017. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:26.957577 26367 tablet_copy_client.cc:806] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: tablet copy: Starting download of 0 data blocks...
I20261118 03:06:26.957711 26367 tablet_copy_client.cc:670] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: tablet copy: Starting download of 1 WAL segments...
I20261118 03:06:26.958509 26367 tablet_copy_client.cc:538] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261118 03:06:26.959175 26367 tablet_bootstrap.cc:492] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: Bootstrap starting.
I20261118 03:06:26.960258 26367 log.cc:826] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:26.961050 26367 tablet_bootstrap.cc:492] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 03:06:26.961239 26367 tablet_bootstrap.cc:492] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: Bootstrap complete.
I20261118 03:06:26.961337 26367 ts_tablet_manager.cc:1449] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20261118 03:06:26.962409 26367 raft_consensus.cc:360] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: true } }
I20261118 03:06:26.962548 26367 raft_consensus.cc:748] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ed230c02f7824013bf487c770d266ad3, State: Initialized, Role: LEARNER
I20261118 03:06:26.962610 26367 consensus_queue.cc:261] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: true } }
I20261118 03:06:26.962831 26367 ts_tablet_manager.cc:1480] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:26.963119 26299 tablet_copy_service.cc:342] P cc92bc45b19d4fc0a95f6cf24766eaeb: Request end of tablet copy session ed230c02f7824013bf487c770d266ad3-0a463bcede924fc883b321026eaf2017 received from {username='slave'} at 127.24.137.129:38725
I20261118 03:06:26.963181 26299 tablet_copy_service.cc:434] P cc92bc45b19d4fc0a95f6cf24766eaeb: ending tablet copy session ed230c02f7824013bf487c770d266ad3-0a463bcede924fc883b321026eaf2017 on tablet 0a463bcede924fc883b321026eaf2017 with peer ed230c02f7824013bf487c770d266ad3
I20261118 03:06:27.383564 25886 raft_consensus.cc:1227] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20261118 03:06:27.852761 26373 raft_consensus.cc:1074] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb: attempting to promote NON_VOTER ed230c02f7824013bf487c770d266ad3 to VOTER
I20261118 03:06:27.853006 26373 consensus_queue.cc:238] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: false } }
I20261118 03:06:27.853595 26148 raft_consensus.cc:1285] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [term 1 FOLLOWER]: Refusing update from remote peer cc92bc45b19d4fc0a95f6cf24766eaeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20261118 03:06:27.853600 26017 raft_consensus.cc:1285] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [term 1 FOLLOWER]: Refusing update from remote peer cc92bc45b19d4fc0a95f6cf24766eaeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20261118 03:06:27.853601 25886 raft_consensus.cc:1285] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3 [term 1 LEARNER]: Refusing update from remote peer cc92bc45b19d4fc0a95f6cf24766eaeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20261118 03:06:27.853832 26375 consensus_queue.cc:1049] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20261118 03:06:27.853955 26359 consensus_queue.cc:1049] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20261118 03:06:27.854080 26373 consensus_queue.cc:1049] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20261118 03:06:27.854863 26373 raft_consensus.cc:2983] T 0a463bcede924fc883b321026eaf2017 P cc92bc45b19d4fc0a95f6cf24766eaeb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ed230c02f7824013bf487c770d266ad3 (127.24.137.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: false } } }
I20261118 03:06:27.854938 26148 raft_consensus.cc:2983] T 0a463bcede924fc883b321026eaf2017 P 44c9837255df4c0da68aa6475c9458d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ed230c02f7824013bf487c770d266ad3 (127.24.137.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: false } } }
I20261118 03:06:27.855101 25886 raft_consensus.cc:2983] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ed230c02f7824013bf487c770d266ad3 (127.24.137.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: false } } }
I20261118 03:06:27.856143 25741 catalog_manager.cc:5840] T 0a463bcede924fc883b321026eaf2017 P ed230c02f7824013bf487c770d266ad3 reported cstate change: config changed from index 2 to 3, ed230c02f7824013bf487c770d266ad3 (127.24.137.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: false } } }
I20261118 03:06:27.856215 26017 raft_consensus.cc:2983] T 0a463bcede924fc883b321026eaf2017 P b3216cd0da36498b85fefc2a5f0cbe7e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ed230c02f7824013bf487c770d266ad3 (127.24.137.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44c9837255df4c0da68aa6475c9458d8" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 41879 } } peers { permanent_uuid: "b3216cd0da36498b85fefc2a5f0cbe7e" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 46425 } } peers { permanent_uuid: "cc92bc45b19d4fc0a95f6cf24766eaeb" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 43763 } } peers { permanent_uuid: "ed230c02f7824013bf487c770d266ad3" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 34337 } attrs { promote: false } } }
I20261118 03:06:27.904098 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 25804
I20261118 03:06:27.908339 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 25935
I20261118 03:06:27.912427 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 26066
I20261118 03:06:27.916545 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 26197
I20261118 03:06:27.920452 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 25712
2026-11-18T03:06:27Z chronyd exiting
[ OK ] RaftConfigChangeITest.TestNonVoterPromotion (2362 ms)
[ RUN ] RaftConfigChangeITest.TestBulkChangeConfig
2026-11-18T03:06:27Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-18T03:06:27Z Disabled control of system clock
I20261118 03:06:27.945235 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:34225
--webserver_interface=127.24.137.190
--webserver_port=0
--builtin_ntp_servers=127.24.137.148:37119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.137.190:34225
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261118 03:06:28.018905 26388 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:28.019068 26388 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:28.019085 26388 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:28.020689 26388 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 03:06:28.020745 26388 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:28.020757 26388 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 03:06:28.020768 26388 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 03:06:28.022459 26388 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.24.137.190:34225
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:34225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.137.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.26388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:28.022651 26388 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:28.022821 26388 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:28.025049 26394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:28.024994 26393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:28.025137 26388 server_base.cc:1061] running on GCE node
W20261118 03:06:28.024991 26396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:28.025503 26388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:28.025727 26388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:28.026852 26388 hybrid_clock.cc:648] HybridClock initialized: now 1794971188026839 us; error 28 us; skew 500 ppm
I20261118 03:06:28.027832 26388 webserver.cc:577] Webserver started at http://127.24.137.190:43935/ using document root <none> and password file <none>
I20261118 03:06:28.028004 26388 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:28.028044 26388 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:28.028126 26388 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:28.028903 26388 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/data/instance:
uuid: "cfbe11a1560149259a44ed9d3e662da3"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.029191 26388 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/wal/instance:
uuid: "cfbe11a1560149259a44ed9d3e662da3"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.030267 26388 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:28.030853 26402 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.031000 26388 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:28.031056 26388 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/wal
uuid: "cfbe11a1560149259a44ed9d3e662da3"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.031108 26388 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:28.048017 26388 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:28.048316 26388 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:28.048418 26388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:28.052058 26388 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.190:34225
I20261118 03:06:28.052110 26454 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.190:34225 every 8 connection(s)
I20261118 03:06:28.052408 26388 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/data/info.pb
I20261118 03:06:28.052923 26455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:28.055202 26455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3: Bootstrap starting.
I20261118 03:06:28.055763 26455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:28.055989 26455 log.cc:826] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:28.056521 26455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3: No bootstrap required, opened a new log
I20261118 03:06:28.057775 26455 raft_consensus.cc:360] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbe11a1560149259a44ed9d3e662da3" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 34225 } }
I20261118 03:06:28.057894 26455 raft_consensus.cc:386] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:28.057912 26455 raft_consensus.cc:748] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfbe11a1560149259a44ed9d3e662da3, State: Initialized, Role: FOLLOWER
I20261118 03:06:28.058032 26455 consensus_queue.cc:261] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbe11a1560149259a44ed9d3e662da3" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 34225 } }
I20261118 03:06:28.058094 26455 raft_consensus.cc:400] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261118 03:06:28.058149 26455 raft_consensus.cc:499] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261118 03:06:28.058190 26455 raft_consensus.cc:3096] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:28.058686 26455 raft_consensus.cc:521] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbe11a1560149259a44ed9d3e662da3" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 34225 } }
I20261118 03:06:28.058789 26455 leader_election.cc:304] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfbe11a1560149259a44ed9d3e662da3; no voters:
I20261118 03:06:28.058956 26455 leader_election.cc:290] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261118 03:06:28.058985 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 26388
I20261118 03:06:28.059015 26458 raft_consensus.cc:2826] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:28.059080 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/master-0/wal/instance
I20261118 03:06:28.059165 26458 raft_consensus.cc:705] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [term 1 LEADER]: Becoming Leader. State: Replica: cfbe11a1560149259a44ed9d3e662da3, State: Running, Role: LEADER
I20261118 03:06:28.059190 26455 sys_catalog.cc:565] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 03:06:28.059309 26458 consensus_queue.cc:238] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbe11a1560149259a44ed9d3e662da3" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 34225 } }
I20261118 03:06:28.059823 26459 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfbe11a1560149259a44ed9d3e662da3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbe11a1560149259a44ed9d3e662da3" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 34225 } } }
I20261118 03:06:28.059924 26459 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [sys.catalog]: This master's current role is: LEADER
I20261118 03:06:28.059821 26460 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfbe11a1560149259a44ed9d3e662da3. Latest consensus state: current_term: 1 leader_uuid: "cfbe11a1560149259a44ed9d3e662da3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbe11a1560149259a44ed9d3e662da3" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 34225 } } }
I20261118 03:06:28.060158 26460 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3 [sys.catalog]: This master's current role is: LEADER
I20261118 03:06:28.060180 26468 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 03:06:28.060668 26468 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 03:06:28.062136 26468 catalog_manager.cc:1398] Generated new cluster ID: c984215dc5574c94a1b8c343588e1cfb
I20261118 03:06:28.062186 26468 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 03:06:28.073045 26468 catalog_manager.cc:1421] Generated new certificate authority record
I20261118 03:06:28.073786 26468 catalog_manager.cc:1555] Loading token signing keys...
I20261118 03:06:28.079383 26468 catalog_manager.cc:6221] T 00000000000000000000000000000000 P cfbe11a1560149259a44ed9d3e662da3: Generated new TSK 0
I20261118 03:06:28.079548 26468 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 03:06:28.080616 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.129:0
--local_ip_for_outbound_sockets=127.24.137.129
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:34225
--builtin_ntp_servers=127.24.137.148:37119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261118 03:06:28.170668 26479 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261118 03:06:28.170843 26479 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:28.170866 26479 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:28.170892 26479 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:28.172479 26479 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:28.172551 26479 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.129
I20261118 03:06:28.174211 26479 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.26479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:28.174450 26479 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:28.174634 26479 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:28.176966 26485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:28.177071 26479 server_base.cc:1061] running on GCE node
W20261118 03:06:28.176966 26484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:28.177137 26487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:28.177372 26479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:28.177587 26479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:28.178717 26479 hybrid_clock.cc:648] HybridClock initialized: now 1794971188178703 us; error 40 us; skew 500 ppm
I20261118 03:06:28.179733 26479 webserver.cc:577] Webserver started at http://127.24.137.129:32781/ using document root <none> and password file <none>
I20261118 03:06:28.179929 26479 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:28.179971 26479 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:28.180056 26479 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:28.180860 26479 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/data/instance:
uuid: "e203afa4f2064ca9b76521c57962d359"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.181166 26479 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/wal/instance:
uuid: "e203afa4f2064ca9b76521c57962d359"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.182374 26479 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261118 03:06:28.183039 26493 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.183197 26479 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:28.183266 26479 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/wal
uuid: "e203afa4f2064ca9b76521c57962d359"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.183327 26479 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:28.210078 26479 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:28.210379 26479 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:28.210497 26479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:28.210718 26479 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:28.211061 26479 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:28.211094 26479 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.211119 26479 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:28.211144 26479 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.218396 26479 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.129:43493
I20261118 03:06:28.218461 26606 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.129:43493 every 8 connection(s)
I20261118 03:06:28.218798 26479 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/data/info.pb
I20261118 03:06:28.223447 26607 heartbeater.cc:345] Connected to a master server at 127.24.137.190:34225
I20261118 03:06:28.223557 26607 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:28.223745 26607 heartbeater.cc:508] Master 127.24.137.190:34225 requested a full tablet report, sending...
I20261118 03:06:28.224265 26419 ts_manager.cc:210] Registered new tserver with Master: e203afa4f2064ca9b76521c57962d359 (127.24.137.129:43493)
I20261118 03:06:28.224934 26419 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.129:40271
I20261118 03:06:28.225654 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 26479
I20261118 03:06:28.225740 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-0/wal/instance
I20261118 03:06:28.226838 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.130:0
--local_ip_for_outbound_sockets=127.24.137.130
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:34225
--builtin_ntp_servers=127.24.137.148:37119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261118 03:06:28.303553 26610 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261118 03:06:28.303725 26610 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:28.303748 26610 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:28.303774 26610 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:28.305356 26610 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:28.305428 26610 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.130
I20261118 03:06:28.307053 26610 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.26610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:28.307281 26610 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:28.307453 26610 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:28.309723 26618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:28.309731 26615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:28.309799 26616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:28.309988 26610 server_base.cc:1061] running on GCE node
I20261118 03:06:28.310119 26610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:28.310316 26610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:28.311437 26610 hybrid_clock.cc:648] HybridClock initialized: now 1794971188311412 us; error 43 us; skew 500 ppm
I20261118 03:06:28.312474 26610 webserver.cc:577] Webserver started at http://127.24.137.130:35291/ using document root <none> and password file <none>
I20261118 03:06:28.312656 26610 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:28.312696 26610 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:28.312786 26610 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:28.313619 26610 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/data/instance:
uuid: "dac2779fd5fc486fa94c4070803b6fb1"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.313926 26610 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/wal/instance:
uuid: "dac2779fd5fc486fa94c4070803b6fb1"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.314993 26610 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:28.315658 26624 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.315828 26610 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:28.315888 26610 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/wal
uuid: "dac2779fd5fc486fa94c4070803b6fb1"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.315943 26610 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:28.327276 26610 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:28.327556 26610 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:28.327670 26610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:28.327867 26610 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:28.328176 26610 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:28.328207 26610 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.328236 26610 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:28.328253 26610 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.333876 26610 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.130:37585
I20261118 03:06:28.333918 26737 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.130:37585 every 8 connection(s)
I20261118 03:06:28.334252 26610 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/data/info.pb
I20261118 03:06:28.338371 26738 heartbeater.cc:345] Connected to a master server at 127.24.137.190:34225
I20261118 03:06:28.338475 26738 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:28.338665 26738 heartbeater.cc:508] Master 127.24.137.190:34225 requested a full tablet report, sending...
I20261118 03:06:28.339041 26419 ts_manager.cc:210] Registered new tserver with Master: dac2779fd5fc486fa94c4070803b6fb1 (127.24.137.130:37585)
I20261118 03:06:28.339423 26419 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.130:56257
I20261118 03:06:28.341101 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 26610
I20261118 03:06:28.341178 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-1/wal/instance
I20261118 03:06:28.342222 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.131:0
--local_ip_for_outbound_sockets=127.24.137.131
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:34225
--builtin_ntp_servers=127.24.137.148:37119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261118 03:06:28.415896 26741 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261118 03:06:28.416062 26741 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:28.416081 26741 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:28.416105 26741 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:28.417677 26741 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:28.417742 26741 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.131
I20261118 03:06:28.419396 26741 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.26741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:28.419605 26741 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:28.419776 26741 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:28.422068 26747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:28.422122 26746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:28.422068 26749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:28.422225 26741 server_base.cc:1061] running on GCE node
I20261118 03:06:28.422444 26741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:28.422645 26741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:28.423766 26741 hybrid_clock.cc:648] HybridClock initialized: now 1794971188423753 us; error 29 us; skew 500 ppm
I20261118 03:06:28.424754 26741 webserver.cc:577] Webserver started at http://127.24.137.131:39781/ using document root <none> and password file <none>
I20261118 03:06:28.424933 26741 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:28.424974 26741 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:28.425064 26741 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:28.425896 26741 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/data/instance:
uuid: "b64a92ca36094830a2118163aef1503d"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.426193 26741 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/wal/instance:
uuid: "b64a92ca36094830a2118163aef1503d"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.427217 26741 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:28.427809 26755 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.427965 26741 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:28.428020 26741 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/wal
uuid: "b64a92ca36094830a2118163aef1503d"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.428072 26741 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:28.442740 26741 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:28.442991 26741 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:28.443132 26741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:28.443326 26741 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:28.443639 26741 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:28.443670 26741 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.443698 26741 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:28.443714 26741 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.449180 26741 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.131:37063
I20261118 03:06:28.449232 26868 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.131:37063 every 8 connection(s)
I20261118 03:06:28.449553 26741 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/data/info.pb
I20261118 03:06:28.453598 26869 heartbeater.cc:345] Connected to a master server at 127.24.137.190:34225
I20261118 03:06:28.453709 26869 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:28.453913 26869 heartbeater.cc:508] Master 127.24.137.190:34225 requested a full tablet report, sending...
I20261118 03:06:28.454322 26419 ts_manager.cc:210] Registered new tserver with Master: b64a92ca36094830a2118163aef1503d (127.24.137.131:37063)
I20261118 03:06:28.454705 26419 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.131:43923
I20261118 03:06:28.456423 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 26741
I20261118 03:06:28.456502 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-2/wal/instance
I20261118 03:06:28.457579 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.132:0
--local_ip_for_outbound_sockets=127.24.137.132
--webserver_interface=127.24.137.132
--webserver_port=0
--tserver_master_addrs=127.24.137.190:34225
--builtin_ntp_servers=127.24.137.148:37119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_prepare_replacement_before_eviction=true with env {}
W20261118 03:06:28.534461 26872 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20261118 03:06:28.534626 26872 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:28.534642 26872 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:28.534660 26872 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:28.536237 26872 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:28.536302 26872 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.132
I20261118 03:06:28.537932 26872 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.24.137.132
--webserver_port=0
--tserver_master_addrs=127.24.137.190:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.26872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:28.538117 26872 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:28.538267 26872 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:28.540639 26880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:28.540768 26877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:28.540690 26872 server_base.cc:1061] running on GCE node
W20261118 03:06:28.540643 26878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:28.540993 26872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:28.541157 26872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:28.542274 26872 hybrid_clock.cc:648] HybridClock initialized: now 1794971188542259 us; error 38 us; skew 500 ppm
I20261118 03:06:28.543231 26872 webserver.cc:577] Webserver started at http://127.24.137.132:33841/ using document root <none> and password file <none>
I20261118 03:06:28.543453 26872 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:28.543497 26872 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:28.543570 26872 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:28.544348 26872 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/data/instance:
uuid: "8c9259ec13a84c40acb50b9b28bba4a6"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.544615 26872 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/wal/instance:
uuid: "8c9259ec13a84c40acb50b9b28bba4a6"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.545665 26872 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20261118 03:06:28.546281 26886 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.546419 26872 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.546478 26872 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/wal
uuid: "8c9259ec13a84c40acb50b9b28bba4a6"
format_stamp: "Formatted at 2026-11-18 03:06:28 on dist-test-slave-k60w"
I20261118 03:06:28.546530 26872 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:28.559603 26872 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:28.559881 26872 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:28.559996 26872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:28.560184 26872 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:28.560490 26872 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:28.560529 26872 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.560567 26872 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:28.560607 26872 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:28.566205 26872 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.132:33585
I20261118 03:06:28.566274 26999 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.132:33585 every 8 connection(s)
I20261118 03:06:28.566546 26872 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/data/info.pb
I20261118 03:06:28.570657 27000 heartbeater.cc:345] Connected to a master server at 127.24.137.190:34225
I20261118 03:06:28.570753 27000 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:28.570940 27000 heartbeater.cc:508] Master 127.24.137.190:34225 requested a full tablet report, sending...
I20261118 03:06:28.571316 26419 ts_manager.cc:210] Registered new tserver with Master: 8c9259ec13a84c40acb50b9b28bba4a6 (127.24.137.132:33585)
I20261118 03:06:28.571676 26419 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.132:37637
I20261118 03:06:28.571770 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 26872
I20261118 03:06:28.571862 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.RaftConfigChangeITest.TestBulkChangeConfig.1794971175865063-25126-0/minicluster-data/ts-3/wal/instance
I20261118 03:06:28.572789 25126 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261118 03:06:28.582697 26419 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48958:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 03:06:28.589046 26672 tablet_service.cc:1500] Processing CreateTablet for tablet 9682d726b01e492898884101fd85f598 (DEFAULT_TABLE table=test-workload [id=38d11d46fd3c4b86ade51ac2b6c01e12]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:28.589370 26672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9682d726b01e492898884101fd85f598. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:28.589361 26541 tablet_service.cc:1500] Processing CreateTablet for tablet 9682d726b01e492898884101fd85f598 (DEFAULT_TABLE table=test-workload [id=38d11d46fd3c4b86ade51ac2b6c01e12]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:28.589623 26541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9682d726b01e492898884101fd85f598. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:28.589645 26934 tablet_service.cc:1500] Processing CreateTablet for tablet 9682d726b01e492898884101fd85f598 (DEFAULT_TABLE table=test-workload [id=38d11d46fd3c4b86ade51ac2b6c01e12]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:28.589865 26934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9682d726b01e492898884101fd85f598. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:28.591516 27022 tablet_bootstrap.cc:492] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359: Bootstrap starting.
I20261118 03:06:28.592106 27022 tablet_bootstrap.cc:654] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:28.592101 27024 tablet_bootstrap.cc:492] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6: Bootstrap starting.
I20261118 03:06:28.592181 27023 tablet_bootstrap.cc:492] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1: Bootstrap starting.
I20261118 03:06:28.592438 27022 log.cc:826] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:28.592689 27024 tablet_bootstrap.cc:654] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:28.592795 27023 tablet_bootstrap.cc:654] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:28.592944 27024 log.cc:826] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:28.592979 27022 tablet_bootstrap.cc:492] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359: No bootstrap required, opened a new log
I20261118 03:06:28.593031 27022 ts_tablet_manager.cc:1449] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261118 03:06:28.593036 27023 log.cc:826] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:28.593484 27024 tablet_bootstrap.cc:492] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6: No bootstrap required, opened a new log
I20261118 03:06:28.593570 27024 ts_tablet_manager.cc:1449] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261118 03:06:28.593632 27023 tablet_bootstrap.cc:492] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1: No bootstrap required, opened a new log
I20261118 03:06:28.593698 27023 ts_tablet_manager.cc:1449] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261118 03:06:28.594259 27022 raft_consensus.cc:360] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } }
I20261118 03:06:28.594357 27022 raft_consensus.cc:748] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e203afa4f2064ca9b76521c57962d359, State: Initialized, Role: FOLLOWER
I20261118 03:06:28.594398 27022 consensus_queue.cc:261] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } }
I20261118 03:06:28.594583 27022 ts_tablet_manager.cc:1480] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 03:06:28.594683 26607 heartbeater.cc:500] Master 127.24.137.190:34225 was elected leader, sending a full tablet report...
I20261118 03:06:28.595074 27023 raft_consensus.cc:360] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } }
I20261118 03:06:28.595060 27024 raft_consensus.cc:360] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } }
I20261118 03:06:28.595177 27023 raft_consensus.cc:748] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dac2779fd5fc486fa94c4070803b6fb1, State: Initialized, Role: FOLLOWER
I20261118 03:06:28.595177 27024 raft_consensus.cc:748] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9259ec13a84c40acb50b9b28bba4a6, State: Initialized, Role: FOLLOWER
I20261118 03:06:28.595237 27024 consensus_queue.cc:261] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } }
I20261118 03:06:28.595237 27023 consensus_queue.cc:261] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } }
I20261118 03:06:28.595443 27023 ts_tablet_manager.cc:1480] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 03:06:28.595443 27024 ts_tablet_manager.cc:1480] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 03:06:28.595486 27000 heartbeater.cc:500] Master 127.24.137.190:34225 was elected leader, sending a full tablet report...
I20261118 03:06:28.595530 26738 heartbeater.cc:500] Master 127.24.137.190:34225 was elected leader, sending a full tablet report...
I20261118 03:06:28.608112 26954 tablet_service.cc:2033] Received Run Leader Election RPC: tablet_id: "9682d726b01e492898884101fd85f598"
dest_uuid: "8c9259ec13a84c40acb50b9b28bba4a6"
from {username='slave'} at 127.0.0.1:38130
I20261118 03:06:28.608225 26954 raft_consensus.cc:499] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20261118 03:06:28.608269 26954 raft_consensus.cc:3096] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:28.608927 26954 raft_consensus.cc:521] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } }
I20261118 03:06:28.609213 26954 leader_election.cc:290] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [CANDIDATE]: Term 1 election: Requested vote from peers e203afa4f2064ca9b76521c57962d359 (127.24.137.129:43493), dac2779fd5fc486fa94c4070803b6fb1 (127.24.137.130:37585)
I20261118 03:06:28.612265 26561 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "9682d726b01e492898884101fd85f598" candidate_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e203afa4f2064ca9b76521c57962d359"
I20261118 03:06:28.612313 26692 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "9682d726b01e492898884101fd85f598" candidate_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dac2779fd5fc486fa94c4070803b6fb1"
I20261118 03:06:28.612397 26561 raft_consensus.cc:3096] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:28.612421 26692 raft_consensus.cc:3096] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:28.613202 26692 raft_consensus.cc:2490] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c9259ec13a84c40acb50b9b28bba4a6 in term 1.
I20261118 03:06:28.613206 26561 raft_consensus.cc:2490] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c9259ec13a84c40acb50b9b28bba4a6 in term 1.
I20261118 03:06:28.613436 26890 leader_election.cc:304] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c9259ec13a84c40acb50b9b28bba4a6, dac2779fd5fc486fa94c4070803b6fb1; no voters:
I20261118 03:06:28.613788 27030 raft_consensus.cc:2826] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:28.613968 27030 raft_consensus.cc:705] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 1 LEADER]: Becoming Leader. State: Replica: 8c9259ec13a84c40acb50b9b28bba4a6, State: Running, Role: LEADER
I20261118 03:06:28.614064 27030 consensus_queue.cc:238] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } }
I20261118 03:06:28.615028 27037 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 03:06:28.615203 26411 catalog_manager.cc:5840] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c9259ec13a84c40acb50b9b28bba4a6 (127.24.137.132). New cstate: current_term: 1 leader_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } health_report { overall_health: HEALTHY } } }
I20261118 03:06:28.616657 27030 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 03:06:28.621773 26521 tablet_service.cc:720] failed op from {username='slave'} at 127.0.0.1:42796: Illegal state: replica e203afa4f2064ca9b76521c57962d359 is not leader of this config: current role FOLLOWER
I20261118 03:06:28.631461 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 26388
I20261118 03:06:28.636349 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:28.636405 25126 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }
I20261118 03:06:28.638000 26954 consensus_queue.cc:238] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }
I20261118 03:06:28.638630 26692 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 FOLLOWER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20261118 03:06:28.638746 26561 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 FOLLOWER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20261118 03:06:28.638860 27037 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20261118 03:06:28.639098 27037 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20261118 03:06:28.639670 27030 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index -1 to 14, VOTER b64a92ca36094830a2118163aef1503d (127.24.137.131) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:28.639808 26561 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index -1 to 14, VOTER b64a92ca36094830a2118163aef1503d (127.24.137.131) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:28.640712 26692 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index -1 to 14, VOTER b64a92ca36094830a2118163aef1503d (127.24.137.131) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
W20261118 03:06:28.641160 27000 heartbeater.cc:648] Failed to heartbeat to 127.24.137.190:34225 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.137.190:34225: connect: Connection refused (error 111)
W20261118 03:06:28.641202 26607 heartbeater.cc:648] Failed to heartbeat to 127.24.137.190:34225 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.137.190:34225: connect: Connection refused (error 111)
W20261118 03:06:28.642114 26738 heartbeater.cc:648] Failed to heartbeat to 127.24.137.190:34225 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.137.190:34225: connect: Connection refused (error 111)
I20261118 03:06:28.642293 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:28.642334 25126 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
I20261118 03:06:28.642369 25126 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } }
I20261118 03:06:28.642656 26954 consensus_queue.cc:238] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }
I20261118 03:06:28.643074 26692 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 FOLLOWER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20261118 03:06:28.643246 27030 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20261118 03:06:28.643253 26888 consensus_peers.cc:597] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 -> Peer b64a92ca36094830a2118163aef1503d (127.24.137.131:37063): Couldn't send request to peer b64a92ca36094830a2118163aef1503d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9682d726b01e492898884101fd85f598. This is attempt 1: this message will repeat every 5th retry.
I20261118 03:06:28.643309 26561 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 FOLLOWER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20261118 03:06:28.643489 27030 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20261118 03:06:28.643826 27039 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index 14 to 15, e203afa4f2064ca9b76521c57962d359 (127.24.137.129) changed from VOTER to NON_VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:28.643931 26692 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, e203afa4f2064ca9b76521c57962d359 (127.24.137.129) changed from VOTER to NON_VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:28.645150 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:28.645195 25126 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } }
I20261118 03:06:28.645310 26561 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 LEARNER]: Committing config change with OpId 1.15: config changed from index 14 to 15, e203afa4f2064ca9b76521c57962d359 (127.24.137.129) changed from VOTER to NON_VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:28.645517 26954 consensus_queue.cc:238] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }
I20261118 03:06:28.645967 26692 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 FOLLOWER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
W20261118 03:06:28.645995 26888 consensus_peers.cc:597] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 -> Peer b64a92ca36094830a2118163aef1503d (127.24.137.131:37063): Couldn't send request to peer b64a92ca36094830a2118163aef1503d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9682d726b01e492898884101fd85f598. This is attempt 1: this message will repeat every 5th retry.
I20261118 03:06:28.646153 27037 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20261118 03:06:28.646230 26560 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 LEARNER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20261118 03:06:28.646525 27037 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20261118 03:06:28.646705 27039 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 1 LEADER]: Committing config change with OpId 1.16: config changed from index 15 to 16. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:28.646844 26691 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:28.647892 26561 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 LEARNER]: Committing config change with OpId 1.16: config changed from index 15 to 16. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:28.648278 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:28.648319 25126 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
I20261118 03:06:28.648370 25126 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } }
I20261118 03:06:28.648944 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:28.649003 25126 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: NON_VOTER last_known_addr { host: "127.24.137.132" port: 33585 } attrs { promote: false replace: false } }
I20261118 03:06:28.649649 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:28.649684 25126 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: true replace: false } }
I20261118 03:06:28.650213 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:28.650245 25126 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
I20261118 03:06:28.650271 25126 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } }
I20261118 03:06:28.650527 26954 consensus_queue.cc:238] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 1.16, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }
W20261118 03:06:28.650938 26888 consensus_peers.cc:597] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 -> Peer b64a92ca36094830a2118163aef1503d (127.24.137.131:37063): Couldn't send request to peer b64a92ca36094830a2118163aef1503d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9682d726b01e492898884101fd85f598. This is attempt 1: this message will repeat every 5th retry.
I20261118 03:06:29.168509 27060 ts_tablet_manager.cc:979] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: Initiating tablet copy from peer 8c9259ec13a84c40acb50b9b28bba4a6 (127.24.137.132:33585)
I20261118 03:06:29.168690 27060 tablet_copy_client.cc:323] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: tablet copy: Beginning tablet copy session from remote peer at address 127.24.137.132:33585
I20261118 03:06:29.171295 26974 tablet_copy_service.cc:140] P 8c9259ec13a84c40acb50b9b28bba4a6: Received BeginTabletCopySession request for tablet 9682d726b01e492898884101fd85f598 from peer b64a92ca36094830a2118163aef1503d ({username='slave'} at 127.24.137.131:59637)
I20261118 03:06:29.171373 26974 tablet_copy_service.cc:161] P 8c9259ec13a84c40acb50b9b28bba4a6: Beginning new tablet copy session on tablet 9682d726b01e492898884101fd85f598 from peer b64a92ca36094830a2118163aef1503d at {username='slave'} at 127.24.137.131:59637: session id = b64a92ca36094830a2118163aef1503d-9682d726b01e492898884101fd85f598
I20261118 03:06:29.171880 26974 tablet_copy_source_session.cc:215] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6: Tablet Copy: opened 0 blocks and 1 log segments
I20261118 03:06:29.172646 27060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9682d726b01e492898884101fd85f598. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:29.174600 27060 tablet_copy_client.cc:806] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: tablet copy: Starting download of 0 data blocks...
I20261118 03:06:29.174734 27060 tablet_copy_client.cc:670] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: tablet copy: Starting download of 1 WAL segments...
I20261118 03:06:29.175585 27060 tablet_copy_client.cc:538] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261118 03:06:29.176218 27060 tablet_bootstrap.cc:492] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: Bootstrap starting.
I20261118 03:06:29.177619 27060 log.cc:826] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:29.180646 27060 tablet_bootstrap.cc:492] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=16 ignored=0} inserts{seen=600 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20261118 03:06:29.180861 27060 tablet_bootstrap.cc:492] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: Bootstrap complete.
I20261118 03:06:29.180975 27060 ts_tablet_manager.cc:1449] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20261118 03:06:29.181782 27060 raft_consensus.cc:360] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }
I20261118 03:06:29.181938 27060 raft_consensus.cc:748] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b64a92ca36094830a2118163aef1503d, State: Initialized, Role: FOLLOWER
I20261118 03:06:29.181988 27060 consensus_queue.cc:261] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 1.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }
I20261118 03:06:29.182185 27060 ts_tablet_manager.cc:1480] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20261118 03:06:29.182262 26869 heartbeater.cc:500] Master 127.24.137.190:34225 was elected leader, sending a full tablet report...
I20261118 03:06:29.182565 26974 tablet_copy_service.cc:342] P 8c9259ec13a84c40acb50b9b28bba4a6: Request end of tablet copy session b64a92ca36094830a2118163aef1503d-9682d726b01e492898884101fd85f598 received from {username='slave'} at 127.24.137.131:59637
I20261118 03:06:29.182631 26974 tablet_copy_service.cc:434] P 8c9259ec13a84c40acb50b9b28bba4a6: ending tablet copy session b64a92ca36094830a2118163aef1503d-9682d726b01e492898884101fd85f598 on tablet 9682d726b01e492898884101fd85f598 with peer b64a92ca36094830a2118163aef1503d
W20261118 03:06:29.182782 26869 heartbeater.cc:648] Failed to heartbeat to 127.24.137.190:34225 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.137.190:34225: connect: Connection refused (error 111)
I20261118 03:06:29.625422 26822 raft_consensus.cc:1227] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.16->[1.17-1.17] Dedup: 1.17->[]
I20261118 03:06:29.625978 27059 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 1 LEADER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER dac2779fd5fc486fa94c4070803b6fb1 (127.24.137.130) evicted, NON_VOTER e203afa4f2064ca9b76521c57962d359 (127.24.137.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:29.626175 26822 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER dac2779fd5fc486fa94c4070803b6fb1 (127.24.137.130) evicted, NON_VOTER e203afa4f2064ca9b76521c57962d359 (127.24.137.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:29.627444 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:29.627504 25126 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
I20261118 03:06:29.627565 25126 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } }
I20261118 03:06:29.628105 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:29.628146 25126 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
I20261118 03:06:29.628399 26954 consensus_queue.cc:238] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 1.17, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
I20261118 03:06:29.628829 26561 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 LEARNER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20261118 03:06:29.628845 26822 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d [term 1 FOLLOWER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20261118 03:06:29.629043 27059 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20261118 03:06:29.629151 27066 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20261118 03:06:29.629663 26561 raft_consensus.cc:879] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }.
Pending config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }:
New config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
W20261118 03:06:29.629712 26561 raft_consensus.cc:1590] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 18' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }.
Pending config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }:
New config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
I20261118 03:06:29.629792 26561 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER dac2779fd5fc486fa94c4070803b6fb1 (127.24.137.130) evicted, NON_VOTER e203afa4f2064ca9b76521c57962d359 (127.24.137.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:29.629982 27059 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 1 LEADER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER e203afa4f2064ca9b76521c57962d359 (127.24.137.129) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } }
I20261118 03:06:29.630106 26822 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER e203afa4f2064ca9b76521c57962d359 (127.24.137.129) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } }
I20261118 03:06:29.630757 26561 raft_consensus.cc:1491] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20261118 03:06:29.630861 26561 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER e203afa4f2064ca9b76521c57962d359 (127.24.137.129) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } }
I20261118 03:06:29.631474 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:29.631508 25126 raft_config_change-itest.cc:351] type: ADD_PEER peer { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } }
I20261118 03:06:29.631785 26954 consensus_queue.cc:238] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 1.18, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } }
I20261118 03:06:29.632239 26822 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d [term 1 FOLLOWER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20261118 03:06:29.632233 26561 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 FOLLOWER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20261118 03:06:29.632280 26691 raft_consensus.cc:1285] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 FOLLOWER]: Refusing update from remote peer 8c9259ec13a84c40acb50b9b28bba4a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20261118 03:06:29.632391 27059 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20261118 03:06:29.632490 27066 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.000s
I20261118 03:06:29.632696 27070 consensus_queue.cc:1049] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20261118 03:06:29.632933 26691 raft_consensus.cc:879] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }.
Pending config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }:
New config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
W20261118 03:06:29.632982 26691 raft_consensus.cc:1590] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 18' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: NON_VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: true replace: false } } peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }.
Pending config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }:
New config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
I20261118 03:06:29.633075 26691 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER dac2779fd5fc486fa94c4070803b6fb1 (127.24.137.130) evicted, NON_VOTER e203afa4f2064ca9b76521c57962d359 (127.24.137.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } }
I20261118 03:06:29.634207 26691 raft_consensus.cc:1491] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20261118 03:06:29.634336 26691 raft_consensus.cc:879] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }.
Pending config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }:
New config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } }
W20261118 03:06:29.634366 26691 raft_consensus.cc:1590] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 19' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } }.
Pending config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }:
New config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } }
I20261118 03:06:29.634392 27070 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P 8c9259ec13a84c40acb50b9b28bba4a6 [term 1 LEADER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER dac2779fd5fc486fa94c4070803b6fb1 (127.24.137.130) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } } }
I20261118 03:06:29.634439 26691 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER e203afa4f2064ca9b76521c57962d359 (127.24.137.129) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } }
I20261118 03:06:29.634508 26561 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P e203afa4f2064ca9b76521c57962d359 [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER dac2779fd5fc486fa94c4070803b6fb1 (127.24.137.130) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } } }
I20261118 03:06:29.635520 26691 raft_consensus.cc:1491] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20261118 03:06:29.635619 26822 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P b64a92ca36094830a2118163aef1503d [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER dac2779fd5fc486fa94c4070803b6fb1 (127.24.137.130) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } } }
I20261118 03:06:29.635660 26691 raft_consensus.cc:2983] T 9682d726b01e492898884101fd85f598 P dac2779fd5fc486fa94c4070803b6fb1 [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER dac2779fd5fc486fa94c4070803b6fb1 (127.24.137.130) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8c9259ec13a84c40acb50b9b28bba4a6" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 33585 } } peers { permanent_uuid: "b64a92ca36094830a2118163aef1503d" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37063 } attrs { promote: false replace: false } } peers { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } } peers { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } } }
I20261118 03:06:29.635891 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:29.635926 25126 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
I20261118 03:06:29.635969 25126 raft_config_change-itest.cc:351] type: REMOVE_PEER peer { permanent_uuid: "dac2779fd5fc486fa94c4070803b6fb1" member_type: UNKNOWN_MEMBER_TYPE last_known_addr { host: "127.24.137.130" port: 37585 } attrs { promote: false replace: false } }
I20261118 03:06:29.636754 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:29.636795 25126 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: false } }
I20261118 03:06:29.637243 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:29.637709 25126 raft_config_change-itest.cc:349] submitting config change with changes:
I20261118 03:06:29.637754 25126 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: false replace: true } }
I20261118 03:06:29.637794 25126 raft_config_change-itest.cc:351] type: MODIFY_PEER peer { permanent_uuid: "e203afa4f2064ca9b76521c57962d359" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 43493 } attrs { promote: true replace: false } }
I20261118 03:06:29.638316 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 26479
I20261118 03:06:29.643345 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 26610
I20261118 03:06:29.647912 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 26741
I20261118 03:06:29.652120 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 26872
2026-11-18T03:06:29Z chronyd exiting
[ OK ] RaftConfigChangeITest.TestBulkChangeConfig (1732 ms)
[----------] 3 tests from RaftConfigChangeITest (13801 ms total)
[----------] 3 tests from SlowTabletBootstrapTest
[ RUN ] SlowTabletBootstrapTest.TestSlowBootstrap
I20261118 03:06:29.671746 25126 ts_itest-base.cc:115] Starting cluster with:
I20261118 03:06:29.671793 25126 ts_itest-base.cc:116] --------------
I20261118 03:06:29.671803 25126 ts_itest-base.cc:117] 4 tablet servers
I20261118 03:06:29.671818 25126 ts_itest-base.cc:118] 3 replicas per TS
I20261118 03:06:29.671838 25126 ts_itest-base.cc:119] --------------
2026-11-18T03:06:29Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-18T03:06:29Z Disabled control of system clock
I20261118 03:06:29.678676 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:43143
--webserver_interface=127.24.137.190
--webserver_port=0
--builtin_ntp_servers=127.24.137.148:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.137.190:43143 with env {}
W20261118 03:06:29.752581 27082 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:29.752743 27082 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:29.752770 27082 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:29.754361 27082 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 03:06:29.754415 27082 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:29.754432 27082 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 03:06:29.754448 27082 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 03:06:29.756069 27082 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.137.190:43143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:43143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.24.137.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.27082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:29.756289 27082 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:29.756445 27082 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:29.758621 27087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:29.758649 27090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:29.758679 27088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:29.758842 27082 server_base.cc:1061] running on GCE node
I20261118 03:06:29.759038 27082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:29.759200 27082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:29.760321 27082 hybrid_clock.cc:648] HybridClock initialized: now 1794971189760309 us; error 28 us; skew 500 ppm
I20261118 03:06:29.761286 27082 webserver.cc:577] Webserver started at http://127.24.137.190:36675/ using document root <none> and password file <none>
I20261118 03:06:29.761468 27082 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:29.761507 27082 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:29.761606 27082 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:29.762414 27082 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "fb7c31efc3a349a08b05138fd2871566"
format_stamp: "Formatted at 2026-11-18 03:06:29 on dist-test-slave-k60w"
I20261118 03:06:29.762722 27082 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "fb7c31efc3a349a08b05138fd2871566"
format_stamp: "Formatted at 2026-11-18 03:06:29 on dist-test-slave-k60w"
I20261118 03:06:29.763725 27082 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:29.764283 27096 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:29.764421 27082 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:29.764490 27082 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
uuid: "fb7c31efc3a349a08b05138fd2871566"
format_stamp: "Formatted at 2026-11-18 03:06:29 on dist-test-slave-k60w"
I20261118 03:06:29.764541 27082 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:29.779265 27082 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:29.779544 27082 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:29.779657 27082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:29.783370 27082 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.190:43143
I20261118 03:06:29.783406 27148 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.190:43143 every 8 connection(s)
I20261118 03:06:29.783713 27082 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20261118 03:06:29.784272 27149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:29.786556 27149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566: Bootstrap starting.
I20261118 03:06:29.787200 27149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:29.787443 27149 log.cc:826] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:29.787997 27149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566: No bootstrap required, opened a new log
I20261118 03:06:29.789158 27149 raft_consensus.cc:360] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7c31efc3a349a08b05138fd2871566" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 43143 } }
I20261118 03:06:29.789278 27149 raft_consensus.cc:386] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:29.789295 27149 raft_consensus.cc:748] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7c31efc3a349a08b05138fd2871566, State: Initialized, Role: FOLLOWER
I20261118 03:06:29.789399 27149 consensus_queue.cc:261] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7c31efc3a349a08b05138fd2871566" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 43143 } }
I20261118 03:06:29.789463 27149 raft_consensus.cc:400] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261118 03:06:29.789490 27149 raft_consensus.cc:499] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261118 03:06:29.789559 27149 raft_consensus.cc:3096] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:29.790064 27149 raft_consensus.cc:521] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7c31efc3a349a08b05138fd2871566" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 43143 } }
I20261118 03:06:29.790159 27149 leader_election.cc:304] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb7c31efc3a349a08b05138fd2871566; no voters:
I20261118 03:06:29.790298 27149 leader_election.cc:290] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261118 03:06:29.790359 27152 raft_consensus.cc:2826] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:29.790493 27152 raft_consensus.cc:705] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [term 1 LEADER]: Becoming Leader. State: Replica: fb7c31efc3a349a08b05138fd2871566, State: Running, Role: LEADER
I20261118 03:06:29.790510 27149 sys_catalog.cc:565] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 03:06:29.790575 27152 consensus_queue.cc:238] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7c31efc3a349a08b05138fd2871566" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 43143 } }
I20261118 03:06:29.790897 27153 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb7c31efc3a349a08b05138fd2871566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7c31efc3a349a08b05138fd2871566" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 43143 } } }
I20261118 03:06:29.790922 27154 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb7c31efc3a349a08b05138fd2871566. Latest consensus state: current_term: 1 leader_uuid: "fb7c31efc3a349a08b05138fd2871566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7c31efc3a349a08b05138fd2871566" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 43143 } } }
I20261118 03:06:29.791014 27153 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [sys.catalog]: This master's current role is: LEADER
I20261118 03:06:29.791019 27154 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566 [sys.catalog]: This master's current role is: LEADER
I20261118 03:06:29.791260 27160 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 03:06:29.791724 27160 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 03:06:29.792274 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 27082
I20261118 03:06:29.792351 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal/instance
I20261118 03:06:29.793291 27160 catalog_manager.cc:1398] Generated new cluster ID: 43423c711c514fa2a9195f158353ac03
I20261118 03:06:29.793334 27160 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 03:06:29.805094 27160 catalog_manager.cc:1421] Generated new certificate authority record
I20261118 03:06:29.806193 27160 catalog_manager.cc:1555] Loading token signing keys...
I20261118 03:06:29.813814 27160 catalog_manager.cc:6221] T 00000000000000000000000000000000 P fb7c31efc3a349a08b05138fd2871566: Generated new TSK 0
I20261118 03:06:29.813997 27160 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 03:06:29.820837 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.129:0
--local_ip_for_outbound_sockets=127.24.137.129
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:43143
--builtin_ntp_servers=127.24.137.148:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261118 03:06:29.900976 27173 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:29.901126 27173 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:29.901140 27173 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:29.901155 27173 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:29.902761 27173 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:29.902807 27173 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:29.902822 27173 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:29.902843 27173 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:29.902854 27173 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.129
I20261118 03:06:29.904474 27173 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:43143
--never_fsync=true
--heap_profile_path=/tmp/kudu.27173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:29.904649 27173 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:29.904820 27173 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:29.907013 27178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:29.907030 27181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:29.907095 27179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:29.907302 27173 server_base.cc:1061] running on GCE node
I20261118 03:06:29.907492 27173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:29.907681 27173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:29.908838 27173 hybrid_clock.cc:648] HybridClock initialized: now 1794971189908823 us; error 28 us; skew 500 ppm
I20261118 03:06:29.910039 27173 webserver.cc:577] Webserver started at http://127.24.137.129:36085/ using document root <none> and password file <none>
I20261118 03:06:29.910234 27173 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:29.910281 27173 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:29.910372 27173 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:29.911374 27173 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "35d3cde7698c4a6b9027a37c775c7052"
format_stamp: "Formatted at 2026-11-18 03:06:29 on dist-test-slave-k60w"
I20261118 03:06:29.911729 27173 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "35d3cde7698c4a6b9027a37c775c7052"
format_stamp: "Formatted at 2026-11-18 03:06:29 on dist-test-slave-k60w"
I20261118 03:06:29.913090 27173 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20261118 03:06:29.913924 27187 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:29.914079 27173 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:29.914149 27173 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "35d3cde7698c4a6b9027a37c775c7052"
format_stamp: "Formatted at 2026-11-18 03:06:29 on dist-test-slave-k60w"
I20261118 03:06:29.914206 27173 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:29.938483 27173 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:29.938695 27173 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:29.938807 27173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:29.939013 27173 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:29.939361 27173 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:29.939397 27173 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:29.939419 27173 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:29.939430 27173 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:29.946012 27173 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.129:46033
I20261118 03:06:29.946062 27300 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.129:46033 every 8 connection(s)
I20261118 03:06:29.946431 27173 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20261118 03:06:29.950650 27301 heartbeater.cc:345] Connected to a master server at 127.24.137.190:43143
I20261118 03:06:29.950800 27301 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:29.950990 27301 heartbeater.cc:508] Master 127.24.137.190:43143 requested a full tablet report, sending...
I20261118 03:06:29.951472 27113 ts_manager.cc:210] Registered new tserver with Master: 35d3cde7698c4a6b9027a37c775c7052 (127.24.137.129:46033)
I20261118 03:06:29.952448 27113 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.129:37855
I20261118 03:06:29.955772 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 27173
I20261118 03:06:29.955876 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20261118 03:06:29.957058 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.130:0
--local_ip_for_outbound_sockets=127.24.137.130
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:43143
--builtin_ntp_servers=127.24.137.148:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261118 03:06:30.031502 27304 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:30.031653 27304 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:30.031667 27304 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:30.031682 27304 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:30.033187 27304 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:30.033236 27304 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:30.033250 27304 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:30.033279 27304 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:30.033298 27304 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.130
I20261118 03:06:30.034848 27304 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:43143
--never_fsync=true
--heap_profile_path=/tmp/kudu.27304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:30.035014 27304 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:30.035149 27304 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:30.037293 27309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:30.037274 27312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:30.037364 27304 server_base.cc:1061] running on GCE node
W20261118 03:06:30.037277 27310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:30.037657 27304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:30.037813 27304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:30.038940 27304 hybrid_clock.cc:648] HybridClock initialized: now 1794971190038926 us; error 27 us; skew 500 ppm
I20261118 03:06:30.039939 27304 webserver.cc:577] Webserver started at http://127.24.137.130:33263/ using document root <none> and password file <none>
I20261118 03:06:30.040105 27304 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:30.040145 27304 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:30.040225 27304 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:30.041007 27304 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "cbd5bd501ddd47adac3298e3bff29f7b"
format_stamp: "Formatted at 2026-11-18 03:06:30 on dist-test-slave-k60w"
I20261118 03:06:30.041308 27304 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "cbd5bd501ddd47adac3298e3bff29f7b"
format_stamp: "Formatted at 2026-11-18 03:06:30 on dist-test-slave-k60w"
I20261118 03:06:30.042366 27304 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261118 03:06:30.043016 27318 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:30.043169 27304 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:30.043226 27304 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "cbd5bd501ddd47adac3298e3bff29f7b"
format_stamp: "Formatted at 2026-11-18 03:06:30 on dist-test-slave-k60w"
I20261118 03:06:30.043301 27304 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:30.056422 27304 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:30.056649 27304 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:30.056752 27304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:30.056934 27304 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:30.057262 27304 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:30.057304 27304 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:30.057333 27304 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:30.057359 27304 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:30.062851 27304 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.130:45183
I20261118 03:06:30.062892 27431 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.130:45183 every 8 connection(s)
I20261118 03:06:30.063182 27304 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261118 03:06:30.067072 27432 heartbeater.cc:345] Connected to a master server at 127.24.137.190:43143
I20261118 03:06:30.067170 27432 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:30.067350 27432 heartbeater.cc:508] Master 127.24.137.190:43143 requested a full tablet report, sending...
I20261118 03:06:30.067701 27113 ts_manager.cc:210] Registered new tserver with Master: cbd5bd501ddd47adac3298e3bff29f7b (127.24.137.130:45183)
I20261118 03:06:30.068109 27113 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.130:54821
I20261118 03:06:30.071439 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 27304
I20261118 03:06:30.071530 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20261118 03:06:30.072643 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.131:0
--local_ip_for_outbound_sockets=127.24.137.131
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:43143
--builtin_ntp_servers=127.24.137.148:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261118 03:06:30.147841 27435 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:30.148041 27435 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:30.148062 27435 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:30.148084 27435 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:30.149742 27435 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:30.149791 27435 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:30.149807 27435 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:30.149829 27435 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:30.149853 27435 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.131
I20261118 03:06:30.151480 27435 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:43143
--never_fsync=true
--heap_profile_path=/tmp/kudu.27435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:30.151678 27435 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:30.151844 27435 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:30.154071 27441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:30.154083 27440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:30.154181 27435 server_base.cc:1061] running on GCE node
W20261118 03:06:30.154258 27443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:30.154403 27435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:30.154580 27435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:30.155699 27435 hybrid_clock.cc:648] HybridClock initialized: now 1794971190155686 us; error 33 us; skew 500 ppm
I20261118 03:06:30.156729 27435 webserver.cc:577] Webserver started at http://127.24.137.131:46237/ using document root <none> and password file <none>
I20261118 03:06:30.156896 27435 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:30.156934 27435 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:30.157015 27435 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:30.157897 27435 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "e42d8d3951f845d4b749678d0ca3406a"
format_stamp: "Formatted at 2026-11-18 03:06:30 on dist-test-slave-k60w"
I20261118 03:06:30.158200 27435 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "e42d8d3951f845d4b749678d0ca3406a"
format_stamp: "Formatted at 2026-11-18 03:06:30 on dist-test-slave-k60w"
I20261118 03:06:30.159230 27435 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:30.159927 27449 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:30.160081 27435 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:30.160135 27435 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "e42d8d3951f845d4b749678d0ca3406a"
format_stamp: "Formatted at 2026-11-18 03:06:30 on dist-test-slave-k60w"
I20261118 03:06:30.160188 27435 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:30.170091 27435 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:30.170290 27435 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:30.170385 27435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:30.170549 27435 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:30.170861 27435 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:30.170892 27435 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:30.170919 27435 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:30.170936 27435 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:30.176327 27435 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.131:43075
I20261118 03:06:30.176395 27562 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.131:43075 every 8 connection(s)
I20261118 03:06:30.176738 27435 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261118 03:06:30.180817 27563 heartbeater.cc:345] Connected to a master server at 127.24.137.190:43143
I20261118 03:06:30.180922 27563 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:30.181103 27563 heartbeater.cc:508] Master 127.24.137.190:43143 requested a full tablet report, sending...
I20261118 03:06:30.181465 27113 ts_manager.cc:210] Registered new tserver with Master: e42d8d3951f845d4b749678d0ca3406a (127.24.137.131:43075)
I20261118 03:06:30.181849 27113 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.131:41207
I20261118 03:06:30.186873 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 27435
I20261118 03:06:30.186969 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20261118 03:06:30.188036 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.132:0
--local_ip_for_outbound_sockets=127.24.137.132
--webserver_interface=127.24.137.132
--webserver_port=0
--tserver_master_addrs=127.24.137.190:43143
--builtin_ntp_servers=127.24.137.148:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261118 03:06:30.262774 27566 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:30.262954 27566 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:30.262969 27566 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:30.262986 27566 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:30.264804 27566 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:30.264860 27566 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:30.264878 27566 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:30.264904 27566 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:30.264938 27566 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.132
I20261118 03:06:30.266670 27566 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.137.132
--webserver_port=0
--tserver_master_addrs=127.24.137.190:43143
--never_fsync=true
--heap_profile_path=/tmp/kudu.27566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:30.266887 27566 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:30.267060 27566 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:30.269297 27572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:30.269362 27566 server_base.cc:1061] running on GCE node
W20261118 03:06:30.269310 27574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:30.269318 27571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:30.269713 27566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:30.269884 27566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:30.271008 27566 hybrid_clock.cc:648] HybridClock initialized: now 1794971190271003 us; error 37 us; skew 500 ppm
I20261118 03:06:30.272033 27566 webserver.cc:577] Webserver started at http://127.24.137.132:43183/ using document root <none> and password file <none>
I20261118 03:06:30.272186 27566 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:30.272220 27566 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:30.272285 27566 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:30.273068 27566 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "23f65c5a6e1745f8abf29f1fb80c6cda"
format_stamp: "Formatted at 2026-11-18 03:06:30 on dist-test-slave-k60w"
I20261118 03:06:30.273330 27566 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "23f65c5a6e1745f8abf29f1fb80c6cda"
format_stamp: "Formatted at 2026-11-18 03:06:30 on dist-test-slave-k60w"
I20261118 03:06:30.274418 27566 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261118 03:06:30.275086 27580 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:30.275235 27566 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:30.275295 27566 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "23f65c5a6e1745f8abf29f1fb80c6cda"
format_stamp: "Formatted at 2026-11-18 03:06:30 on dist-test-slave-k60w"
I20261118 03:06:30.275353 27566 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:30.305088 27566 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:30.305316 27566 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:30.305418 27566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:30.305645 27566 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:30.305961 27566 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:30.305992 27566 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:30.306022 27566 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:30.306047 27566 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:30.311403 27566 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.132:41179
I20261118 03:06:30.311455 27693 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.132:41179 every 8 connection(s)
I20261118 03:06:30.311748 27566 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261118 03:06:30.312038 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 27566
I20261118 03:06:30.312135 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20261118 03:06:30.315867 27694 heartbeater.cc:345] Connected to a master server at 127.24.137.190:43143
I20261118 03:06:30.315969 27694 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:30.316151 27694 heartbeater.cc:508] Master 127.24.137.190:43143 requested a full tablet report, sending...
I20261118 03:06:30.316497 27113 ts_manager.cc:210] Registered new tserver with Master: 23f65c5a6e1745f8abf29f1fb80c6cda (127.24.137.132:41179)
I20261118 03:06:30.316861 27113 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.132:34031
I20261118 03:06:30.323873 25126 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261118 03:06:30.329227 27113 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35364:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20261118 03:06:30.336707 27628 tablet_service.cc:1500] Processing CreateTablet for tablet 1feb954e6c6f47c6b9ba966593f5f1db (DEFAULT_TABLE table=TestTable [id=b0b027d90ad24983adedc3e6ad758436]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:30.336779 27366 tablet_service.cc:1500] Processing CreateTablet for tablet 1feb954e6c6f47c6b9ba966593f5f1db (DEFAULT_TABLE table=TestTable [id=b0b027d90ad24983adedc3e6ad758436]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:30.336841 27235 tablet_service.cc:1500] Processing CreateTablet for tablet 1feb954e6c6f47c6b9ba966593f5f1db (DEFAULT_TABLE table=TestTable [id=b0b027d90ad24983adedc3e6ad758436]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:30.336997 27628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1feb954e6c6f47c6b9ba966593f5f1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:30.337047 27235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1feb954e6c6f47c6b9ba966593f5f1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:30.337049 27366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1feb954e6c6f47c6b9ba966593f5f1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:30.339046 27711 tablet_bootstrap.cc:492] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Bootstrap starting.
I20261118 03:06:30.339474 27712 tablet_bootstrap.cc:492] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052: Bootstrap starting.
I20261118 03:06:30.339648 27711 tablet_bootstrap.cc:654] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:30.339731 27713 tablet_bootstrap.cc:492] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b: Bootstrap starting.
I20261118 03:06:30.339969 27711 log.cc:826] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:30.340036 27712 tablet_bootstrap.cc:654] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:30.340265 27712 log.cc:826] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:30.340302 27713 tablet_bootstrap.cc:654] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:30.340553 27713 log.cc:826] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:30.340652 27711 tablet_bootstrap.cc:492] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: No bootstrap required, opened a new log
I20261118 03:06:30.340723 27711 ts_tablet_manager.cc:1449] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261118 03:06:30.340786 27712 tablet_bootstrap.cc:492] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052: No bootstrap required, opened a new log
I20261118 03:06:30.340853 27712 ts_tablet_manager.cc:1449] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:30.341161 27713 tablet_bootstrap.cc:492] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b: No bootstrap required, opened a new log
I20261118 03:06:30.341212 27713 ts_tablet_manager.cc:1449] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261118 03:06:30.341964 27712 raft_consensus.cc:360] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:30.342062 27712 raft_consensus.cc:386] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:30.342079 27712 raft_consensus.cc:748] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d3cde7698c4a6b9027a37c775c7052, State: Initialized, Role: FOLLOWER
I20261118 03:06:30.342149 27712 consensus_queue.cc:261] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:30.342344 27712 ts_tablet_manager.cc:1480] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:30.342283 27711 raft_consensus.cc:360] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:30.342398 27711 raft_consensus.cc:386] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:30.342414 27301 heartbeater.cc:500] Master 127.24.137.190:43143 was elected leader, sending a full tablet report...
I20261118 03:06:30.342423 27711 raft_consensus.cc:748] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f65c5a6e1745f8abf29f1fb80c6cda, State: Initialized, Role: FOLLOWER
I20261118 03:06:30.342513 27711 consensus_queue.cc:261] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:30.342711 27711 ts_tablet_manager.cc:1480] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 03:06:30.342748 27713 raft_consensus.cc:360] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:30.342844 27694 heartbeater.cc:500] Master 127.24.137.190:43143 was elected leader, sending a full tablet report...
I20261118 03:06:30.342864 27713 raft_consensus.cc:386] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:30.342926 27713 raft_consensus.cc:748] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd5bd501ddd47adac3298e3bff29f7b, State: Initialized, Role: FOLLOWER
I20261118 03:06:30.343034 27713 consensus_queue.cc:261] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:30.343238 27713 ts_tablet_manager.cc:1480] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 03:06:30.343289 27432 heartbeater.cc:500] Master 127.24.137.190:43143 was elected leader, sending a full tablet report...
W20261118 03:06:30.347868 27302 tablet.cc:2441] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261118 03:06:30.363946 27433 tablet.cc:2441] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261118 03:06:30.412240 27695 tablet.cc:2441] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 03:06:30.509466 27719 raft_consensus.cc:499] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 03:06:30.509613 27719 raft_consensus.cc:521] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:30.509944 27719 leader_election.cc:290] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35d3cde7698c4a6b9027a37c775c7052 (127.24.137.129:46033), 23f65c5a6e1745f8abf29f1fb80c6cda (127.24.137.132:41179)
I20261118 03:06:30.512864 27255 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1feb954e6c6f47c6b9ba966593f5f1db" candidate_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35d3cde7698c4a6b9027a37c775c7052" is_pre_election: true
I20261118 03:06:30.513016 27255 raft_consensus.cc:2490] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbd5bd501ddd47adac3298e3bff29f7b in term 0.
I20261118 03:06:30.513226 27319 leader_election.cc:304] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d3cde7698c4a6b9027a37c775c7052, cbd5bd501ddd47adac3298e3bff29f7b; no voters:
I20261118 03:06:30.513218 27648 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1feb954e6c6f47c6b9ba966593f5f1db" candidate_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" is_pre_election: true
I20261118 03:06:30.513337 27648 raft_consensus.cc:2490] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbd5bd501ddd47adac3298e3bff29f7b in term 0.
I20261118 03:06:30.513406 27719 raft_consensus.cc:2826] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 03:06:30.513473 27719 raft_consensus.cc:499] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 03:06:30.513554 27719 raft_consensus.cc:3096] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:30.514214 27719 raft_consensus.cc:521] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:30.514333 27719 leader_election.cc:290] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [CANDIDATE]: Term 1 election: Requested vote from peers 35d3cde7698c4a6b9027a37c775c7052 (127.24.137.129:46033), 23f65c5a6e1745f8abf29f1fb80c6cda (127.24.137.132:41179)
I20261118 03:06:30.514487 27255 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1feb954e6c6f47c6b9ba966593f5f1db" candidate_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35d3cde7698c4a6b9027a37c775c7052"
I20261118 03:06:30.514503 27648 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1feb954e6c6f47c6b9ba966593f5f1db" candidate_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda"
I20261118 03:06:30.514552 27255 raft_consensus.cc:3096] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:30.514576 27648 raft_consensus.cc:3096] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:30.515103 27255 raft_consensus.cc:2490] T 1feb954e6c6f47c6b9ba966593f5f1db P 35d3cde7698c4a6b9027a37c775c7052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbd5bd501ddd47adac3298e3bff29f7b in term 1.
I20261118 03:06:30.515269 27319 leader_election.cc:304] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d3cde7698c4a6b9027a37c775c7052, cbd5bd501ddd47adac3298e3bff29f7b; no voters:
I20261118 03:06:30.515368 27719 raft_consensus.cc:2826] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:30.515513 27719 raft_consensus.cc:705] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [term 1 LEADER]: Becoming Leader. State: Replica: cbd5bd501ddd47adac3298e3bff29f7b, State: Running, Role: LEADER
I20261118 03:06:30.515550 27648 raft_consensus.cc:2490] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbd5bd501ddd47adac3298e3bff29f7b in term 1.
I20261118 03:06:30.515606 27719 consensus_queue.cc:238] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:30.516332 27112 catalog_manager.cc:5840] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b reported cstate change: term changed from 0 to 1, leader changed from <none> to cbd5bd501ddd47adac3298e3bff29f7b (127.24.137.130). New cstate: current_term: 1 leader_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } health_report { overall_health: UNKNOWN } } }
I20261118 03:06:30.516568 27723 consensus_queue.cc:1049] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 03:06:30.516691 27719 consensus_queue.cc:1049] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 03:06:30.559211 25126 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261118 03:06:30.559701 25126 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 35d3cde7698c4a6b9027a37c775c7052 to finish bootstrapping
I20261118 03:06:30.563207 25126 ts_itest-base.cc:246] Waiting for 1 tablets on tserver cbd5bd501ddd47adac3298e3bff29f7b to finish bootstrapping
I20261118 03:06:30.566033 25126 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 23f65c5a6e1745f8abf29f1fb80c6cda to finish bootstrapping
I20261118 03:06:30.568970 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 27566
I20261118 03:06:30.574010 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.132:41179
--local_ip_for_outbound_sockets=127.24.137.132
--tserver_master_addrs=127.24.137.190:43143
--webserver_port=43183
--webserver_interface=127.24.137.132
--builtin_ntp_servers=127.24.137.148:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000
--tablet_bootstrap_inject_latency_ms=4000 with env {}
W20261118 03:06:30.648540 27730 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:30.648690 27730 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:30.648705 27730 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:30.648721 27730 flags.cc:432] Enabled unsafe flag: --tablet_bootstrap_inject_latency_ms=4000
W20261118 03:06:30.648743 27730 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:30.650301 27730 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:30.650346 27730 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:30.650360 27730 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:30.650398 27730 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:30.650424 27730 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.132
I20261118 03:06:30.651988 27730 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.132:41179
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.137.132
--webserver_port=43183
--tserver_master_addrs=127.24.137.190:43143
--tablet_bootstrap_inject_latency_ms=4000
--never_fsync=true
--heap_profile_path=/tmp/kudu.27730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:30.652170 27730 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:30.652343 27730 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:30.654454 27736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:30.654590 27730 server_base.cc:1061] running on GCE node
W20261118 03:06:30.654479 27738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:30.654488 27735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:30.654845 27730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:30.655032 27730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:30.656159 27730 hybrid_clock.cc:648] HybridClock initialized: now 1794971190656147 us; error 23 us; skew 500 ppm
I20261118 03:06:30.657178 27730 webserver.cc:577] Webserver started at http://127.24.137.132:43183/ using document root <none> and password file <none>
I20261118 03:06:30.657372 27730 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:30.657433 27730 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:30.658565 27730 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:30.659200 27744 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:30.659351 27730 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:30.659412 27730 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "23f65c5a6e1745f8abf29f1fb80c6cda"
format_stamp: "Formatted at 2026-11-18 03:06:30 on dist-test-slave-k60w"
I20261118 03:06:30.659675 27730 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:30.671154 27730 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:30.671378 27730 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:30.671490 27730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:30.671705 27730 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:30.672117 27751 ts_tablet_manager.cc:588] Loading tablet metadata (0/1 complete)
I20261118 03:06:30.672925 27730 ts_tablet_manager.cc:631] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20261118 03:06:30.672973 27730 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20261118 03:06:30.673012 27730 ts_tablet_manager.cc:646] Registering tablets (0/1 complete)
W20261118 03:06:30.673579 27751 ts_tablet_manager.cc:1428] Injecting 4000ms delay in tablet bootstrapping
I20261118 03:06:30.673601 27730 ts_tablet_manager.cc:662] Registered 1 tablets
I20261118 03:06:30.673645 27730 ts_tablet_manager.cc:641] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:30.679522 27730 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.132:41179
I20261118 03:06:30.679586 27858 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.132:41179 every 8 connection(s)
I20261118 03:06:30.679908 27730 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261118 03:06:30.684134 27859 heartbeater.cc:345] Connected to a master server at 127.24.137.190:43143
I20261118 03:06:30.684227 27859 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:30.684430 27859 heartbeater.cc:508] Master 127.24.137.190:43143 requested a full tablet report, sending...
I20261118 03:06:30.684890 27112 ts_manager.cc:210] Re-registered known tserver with Master: 23f65c5a6e1745f8abf29f1fb80c6cda (127.24.137.132:41179)
I20261118 03:06:30.685282 27112 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.132:50791
I20261118 03:06:30.687945 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 27730
W20261118 03:06:30.947243 27319 consensus_peers.cc:597] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b -> Peer 23f65c5a6e1745f8abf29f1fb80c6cda (127.24.137.132:41179): Couldn't send request to peer 23f65c5a6e1745f8abf29f1fb80c6cda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20261118 03:06:31.182504 27563 heartbeater.cc:500] Master 127.24.137.190:43143 was elected leader, sending a full tablet report...
I20261118 03:06:31.686074 27859 heartbeater.cc:500] Master 127.24.137.190:43143 was elected leader, sending a full tablet report...
W20261118 03:06:33.656356 27319 consensus_peers.cc:597] T 1feb954e6c6f47c6b9ba966593f5f1db P cbd5bd501ddd47adac3298e3bff29f7b -> Peer 23f65c5a6e1745f8abf29f1fb80c6cda (127.24.137.132:41179): Couldn't send request to peer 23f65c5a6e1745f8abf29f1fb80c6cda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20261118 03:06:34.673820 27751 tablet_bootstrap.cc:492] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Bootstrap starting.
I20261118 03:06:34.676678 27751 log.cc:826] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:34.677577 27751 tablet_bootstrap.cc:492] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 03:06:34.677794 27751 tablet_bootstrap.cc:492] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Bootstrap complete.
I20261118 03:06:34.677942 27751 ts_tablet_manager.cc:1449] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20261118 03:06:34.678881 27751 raft_consensus.cc:360] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:34.679062 27751 raft_consensus.cc:748] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f65c5a6e1745f8abf29f1fb80c6cda, State: Initialized, Role: FOLLOWER
I20261118 03:06:34.679157 27751 consensus_queue.cc:261] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5bd501ddd47adac3298e3bff29f7b" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 45183 } } peers { permanent_uuid: "35d3cde7698c4a6b9027a37c775c7052" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46033 } } peers { permanent_uuid: "23f65c5a6e1745f8abf29f1fb80c6cda" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 41179 } }
I20261118 03:06:34.679354 27751 ts_tablet_manager.cc:1480] T 1feb954e6c6f47c6b9ba966593f5f1db P 23f65c5a6e1745f8abf29f1fb80c6cda: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:37.692642 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 27173
I20261118 03:06:37.697407 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 27304
I20261118 03:06:37.701682 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 27435
I20261118 03:06:37.707532 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 27730
I20261118 03:06:37.711480 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 27082
2026-11-18T03:06:37Z chronyd exiting
[ OK ] SlowTabletBootstrapTest.TestSlowBootstrap (8059 ms)
[ RUN ] SlowTabletBootstrapTest.TestFallBehind
I20261118 03:06:37.731060 25126 ts_itest-base.cc:115] Starting cluster with:
I20261118 03:06:37.731104 25126 ts_itest-base.cc:116] --------------
I20261118 03:06:37.731114 25126 ts_itest-base.cc:117] 4 tablet servers
I20261118 03:06:37.731124 25126 ts_itest-base.cc:118] 3 replicas per TS
I20261118 03:06:37.731135 25126 ts_itest-base.cc:119] --------------
2026-11-18T03:06:37Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-18T03:06:37Z Disabled control of system clock
I20261118 03:06:37.737763 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:38327
--webserver_interface=127.24.137.190
--webserver_port=0
--builtin_ntp_servers=127.24.137.148:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.137.190:38327 with env {}
W20261118 03:06:37.811216 27877 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:37.811374 27877 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:37.811391 27877 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:37.812873 27877 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 03:06:37.812916 27877 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:37.812928 27877 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 03:06:37.812938 27877 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 03:06:37.814536 27877 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.137.190:38327
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:38327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.24.137.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.27877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:37.814721 27877 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:37.814900 27877 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:37.817343 27882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:37.817420 27877 server_base.cc:1061] running on GCE node
W20261118 03:06:37.817348 27885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:37.817478 27883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:37.817778 27877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:37.817981 27877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:37.819120 27877 hybrid_clock.cc:648] HybridClock initialized: now 1794971197819109 us; error 33 us; skew 500 ppm
I20261118 03:06:37.820032 27877 webserver.cc:577] Webserver started at http://127.24.137.190:45461/ using document root <none> and password file <none>
I20261118 03:06:37.820207 27877 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:37.820241 27877 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:37.820324 27877 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:37.821353 27877 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5"
format_stamp: "Formatted at 2026-11-18 03:06:37 on dist-test-slave-k60w"
I20261118 03:06:37.821692 27877 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5"
format_stamp: "Formatted at 2026-11-18 03:06:37 on dist-test-slave-k60w"
I20261118 03:06:37.822726 27877 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:37.823289 27891 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:37.823441 27877 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:37.823498 27877 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5"
format_stamp: "Formatted at 2026-11-18 03:06:37 on dist-test-slave-k60w"
I20261118 03:06:37.823552 27877 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:37.840996 27877 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:37.841259 27877 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:37.841392 27877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:37.845270 27877 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.190:38327
I20261118 03:06:37.845311 27943 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.190:38327 every 8 connection(s)
I20261118 03:06:37.845687 27877 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20261118 03:06:37.846261 27944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:37.848361 27944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5: Bootstrap starting.
I20261118 03:06:37.848963 27944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:37.849200 27944 log.cc:826] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:37.849745 27944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5: No bootstrap required, opened a new log
I20261118 03:06:37.850907 27944 raft_consensus.cc:360] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 38327 } }
I20261118 03:06:37.851063 27944 raft_consensus.cc:386] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:37.851089 27944 raft_consensus.cc:748] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6b6f5e7a48411eabd4fd01d3d45ac5, State: Initialized, Role: FOLLOWER
I20261118 03:06:37.851090 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 27877
I20261118 03:06:37.851182 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal/instance
I20261118 03:06:37.851197 27944 consensus_queue.cc:261] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 38327 } }
I20261118 03:06:37.851266 27944 raft_consensus.cc:400] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261118 03:06:37.851305 27944 raft_consensus.cc:499] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261118 03:06:37.851343 27944 raft_consensus.cc:3096] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:37.851904 27944 raft_consensus.cc:521] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 38327 } }
I20261118 03:06:37.852012 27944 leader_election.cc:304] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd6b6f5e7a48411eabd4fd01d3d45ac5; no voters:
I20261118 03:06:37.852140 27944 leader_election.cc:290] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20261118 03:06:37.852187 27949 raft_consensus.cc:2826] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:37.852284 27949 raft_consensus.cc:705] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [term 1 LEADER]: Becoming Leader. State: Replica: cd6b6f5e7a48411eabd4fd01d3d45ac5, State: Running, Role: LEADER
I20261118 03:06:37.852361 27944 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 03:06:37.852428 27949 consensus_queue.cc:238] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 38327 } }
I20261118 03:06:37.852773 27949 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd6b6f5e7a48411eabd4fd01d3d45ac5. Latest consensus state: current_term: 1 leader_uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 38327 } } }
I20261118 03:06:37.852851 27949 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [sys.catalog]: This master's current role is: LEADER
I20261118 03:06:37.852787 27950 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd6b6f5e7a48411eabd4fd01d3d45ac5" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 38327 } } }
I20261118 03:06:37.852909 27950 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5 [sys.catalog]: This master's current role is: LEADER
W20261118 03:06:37.854332 27964 catalog_manager.cc:1609] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20261118 03:06:37.854390 27964 catalog_manager.cc:898] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20261118 03:06:37.854437 27955 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 03:06:37.854640 27955 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 03:06:37.856165 27955 catalog_manager.cc:1398] Generated new cluster ID: 6f96a73abafc44db900879f3ccf98cca
I20261118 03:06:37.856218 27955 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 03:06:37.868775 27955 catalog_manager.cc:1421] Generated new certificate authority record
I20261118 03:06:37.869504 27955 catalog_manager.cc:1555] Loading token signing keys...
I20261118 03:06:37.876667 27955 catalog_manager.cc:6221] T 00000000000000000000000000000000 P cd6b6f5e7a48411eabd4fd01d3d45ac5: Generated new TSK 0
I20261118 03:06:37.876850 27955 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 03:06:37.881145 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.129:0
--local_ip_for_outbound_sockets=127.24.137.129
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:38327
--builtin_ntp_servers=127.24.137.148:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261118 03:06:37.955600 27968 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:37.955757 27968 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:37.955777 27968 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:37.955798 27968 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:37.957345 27968 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:37.957396 27968 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:37.957417 27968 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:37.957445 27968 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:37.957463 27968 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.129
I20261118 03:06:37.959372 27968 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:38327
--never_fsync=true
--heap_profile_path=/tmp/kudu.27968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:37.959601 27968 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:37.959784 27968 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:37.962096 27974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:37.962100 27976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:37.962160 27973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:37.962348 27968 server_base.cc:1061] running on GCE node
I20261118 03:06:37.962509 27968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:37.962688 27968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:37.963809 27968 hybrid_clock.cc:648] HybridClock initialized: now 1794971197963786 us; error 41 us; skew 500 ppm
I20261118 03:06:37.964851 27968 webserver.cc:577] Webserver started at http://127.24.137.129:46319/ using document root <none> and password file <none>
I20261118 03:06:37.965027 27968 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:37.965065 27968 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:37.965157 27968 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:37.966008 27968 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "cdb74cb32ea5464eabee31b59c7ab263"
format_stamp: "Formatted at 2026-11-18 03:06:37 on dist-test-slave-k60w"
I20261118 03:06:37.966326 27968 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "cdb74cb32ea5464eabee31b59c7ab263"
format_stamp: "Formatted at 2026-11-18 03:06:37 on dist-test-slave-k60w"
I20261118 03:06:37.967442 27968 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:37.968146 27982 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:37.968318 27968 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:37.968392 27968 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "cdb74cb32ea5464eabee31b59c7ab263"
format_stamp: "Formatted at 2026-11-18 03:06:37 on dist-test-slave-k60w"
I20261118 03:06:37.968451 27968 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:38.017766 27968 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:38.018009 27968 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:38.018111 27968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:38.018301 27968 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:38.018635 27968 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:38.018672 27968 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.018707 27968 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:38.018728 27968 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.024505 27968 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.129:46795
I20261118 03:06:38.024586 28095 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.129:46795 every 8 connection(s)
I20261118 03:06:38.024893 27968 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20261118 03:06:38.025815 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 27968
I20261118 03:06:38.025929 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20261118 03:06:38.027081 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.130:0
--local_ip_for_outbound_sockets=127.24.137.130
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:38327
--builtin_ntp_servers=127.24.137.148:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261118 03:06:38.029493 28096 heartbeater.cc:345] Connected to a master server at 127.24.137.190:38327
I20261118 03:06:38.029608 28096 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:38.029789 28096 heartbeater.cc:508] Master 127.24.137.190:38327 requested a full tablet report, sending...
I20261118 03:06:38.030303 27908 ts_manager.cc:210] Registered new tserver with Master: cdb74cb32ea5464eabee31b59c7ab263 (127.24.137.129:46795)
I20261118 03:06:38.031276 27908 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.129:39563
W20261118 03:06:38.104118 28099 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:38.104274 28099 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:38.104287 28099 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:38.104303 28099 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:38.105890 28099 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:38.105942 28099 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:38.105957 28099 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:38.105978 28099 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:38.105993 28099 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.130
I20261118 03:06:38.107597 28099 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:38327
--never_fsync=true
--heap_profile_path=/tmp/kudu.28099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:38.107769 28099 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:38.107918 28099 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:38.110060 28107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:38.110042 28104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:38.110168 28099 server_base.cc:1061] running on GCE node
W20261118 03:06:38.110042 28105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:38.110369 28099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:38.110549 28099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:38.111672 28099 hybrid_clock.cc:648] HybridClock initialized: now 1794971198111650 us; error 39 us; skew 500 ppm
I20261118 03:06:38.112665 28099 webserver.cc:577] Webserver started at http://127.24.137.130:37831/ using document root <none> and password file <none>
I20261118 03:06:38.112834 28099 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:38.112874 28099 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:38.112955 28099 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:38.113765 28099 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "dd22ed538cf0402f9542416dc9d2ca64"
format_stamp: "Formatted at 2026-11-18 03:06:38 on dist-test-slave-k60w"
I20261118 03:06:38.114053 28099 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "dd22ed538cf0402f9542416dc9d2ca64"
format_stamp: "Formatted at 2026-11-18 03:06:38 on dist-test-slave-k60w"
I20261118 03:06:38.115046 28099 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:38.115679 28113 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.115828 28099 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:38.115885 28099 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "dd22ed538cf0402f9542416dc9d2ca64"
format_stamp: "Formatted at 2026-11-18 03:06:38 on dist-test-slave-k60w"
I20261118 03:06:38.115938 28099 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:38.133651 28099 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:38.133869 28099 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:38.133972 28099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:38.134157 28099 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:38.134474 28099 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:38.134506 28099 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.134539 28099 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:38.134557 28099 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.139763 28099 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.130:38049
I20261118 03:06:38.139806 28226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.130:38049 every 8 connection(s)
I20261118 03:06:38.140106 28099 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261118 03:06:38.141575 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 28099
I20261118 03:06:38.141669 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20261118 03:06:38.142725 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.131:0
--local_ip_for_outbound_sockets=127.24.137.131
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:38327
--builtin_ntp_servers=127.24.137.148:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261118 03:06:38.144485 28227 heartbeater.cc:345] Connected to a master server at 127.24.137.190:38327
I20261118 03:06:38.144575 28227 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:38.144757 28227 heartbeater.cc:508] Master 127.24.137.190:38327 requested a full tablet report, sending...
I20261118 03:06:38.145102 27908 ts_manager.cc:210] Registered new tserver with Master: dd22ed538cf0402f9542416dc9d2ca64 (127.24.137.130:38049)
I20261118 03:06:38.145521 27908 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.130:38489
W20261118 03:06:38.216853 28230 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:38.217025 28230 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:38.217047 28230 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:38.217069 28230 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:38.218621 28230 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:38.218669 28230 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:38.218690 28230 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:38.218752 28230 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:38.218807 28230 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.131
I20261118 03:06:38.220408 28230 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:38327
--never_fsync=true
--heap_profile_path=/tmp/kudu.28230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:38.220618 28230 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:38.220783 28230 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:38.222956 28238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:38.223026 28235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:38.222971 28236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:38.223047 28230 server_base.cc:1061] running on GCE node
I20261118 03:06:38.223235 28230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:38.223392 28230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:38.224514 28230 hybrid_clock.cc:648] HybridClock initialized: now 1794971198224500 us; error 26 us; skew 500 ppm
I20261118 03:06:38.225584 28230 webserver.cc:577] Webserver started at http://127.24.137.131:34063/ using document root <none> and password file <none>
I20261118 03:06:38.225762 28230 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:38.225800 28230 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:38.225885 28230 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:38.226711 28230 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "8ef3d196e0e04469aaebe0da28a01598"
format_stamp: "Formatted at 2026-11-18 03:06:38 on dist-test-slave-k60w"
I20261118 03:06:38.227031 28230 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "8ef3d196e0e04469aaebe0da28a01598"
format_stamp: "Formatted at 2026-11-18 03:06:38 on dist-test-slave-k60w"
I20261118 03:06:38.228053 28230 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261118 03:06:38.228801 28244 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.228977 28230 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20261118 03:06:38.229033 28230 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "8ef3d196e0e04469aaebe0da28a01598"
format_stamp: "Formatted at 2026-11-18 03:06:38 on dist-test-slave-k60w"
I20261118 03:06:38.229087 28230 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:38.246619 28230 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:38.246837 28230 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:38.246939 28230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:38.247121 28230 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:38.247421 28230 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:38.247452 28230 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.247480 28230 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:38.247498 28230 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.252735 28230 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.131:46181
I20261118 03:06:38.252779 28357 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.131:46181 every 8 connection(s)
I20261118 03:06:38.253074 28230 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261118 03:06:38.256816 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 28230
I20261118 03:06:38.256923 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20261118 03:06:38.256956 28358 heartbeater.cc:345] Connected to a master server at 127.24.137.190:38327
I20261118 03:06:38.257059 28358 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:38.257237 28358 heartbeater.cc:508] Master 127.24.137.190:38327 requested a full tablet report, sending...
I20261118 03:06:38.257622 27907 ts_manager.cc:210] Registered new tserver with Master: 8ef3d196e0e04469aaebe0da28a01598 (127.24.137.131:46181)
I20261118 03:06:38.257985 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.132:0
--local_ip_for_outbound_sockets=127.24.137.132
--webserver_interface=127.24.137.132
--webserver_port=0
--tserver_master_addrs=127.24.137.190:38327
--builtin_ntp_servers=127.24.137.148:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20261118 03:06:38.258018 27907 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.131:51071
W20261118 03:06:38.333897 28361 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:38.334052 28361 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:38.334066 28361 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:38.334081 28361 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:38.335667 28361 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:38.335712 28361 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:38.335726 28361 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:38.335749 28361 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:38.335774 28361 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.132
I20261118 03:06:38.337361 28361 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.137.132
--webserver_port=0
--tserver_master_addrs=127.24.137.190:38327
--never_fsync=true
--heap_profile_path=/tmp/kudu.28361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:38.337567 28361 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:38.337733 28361 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:38.340009 28367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:38.340046 28361 server_base.cc:1061] running on GCE node
W20261118 03:06:38.339984 28369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:38.339994 28366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:38.340294 28361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:38.340466 28361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:38.341583 28361 hybrid_clock.cc:648] HybridClock initialized: now 1794971198341561 us; error 33 us; skew 500 ppm
I20261118 03:06:38.342571 28361 webserver.cc:577] Webserver started at http://127.24.137.132:34881/ using document root <none> and password file <none>
I20261118 03:06:38.342741 28361 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:38.342787 28361 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:38.342875 28361 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:38.343652 28361 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "c1d212a9213d4d1f88747c422dd7c8c7"
format_stamp: "Formatted at 2026-11-18 03:06:38 on dist-test-slave-k60w"
I20261118 03:06:38.343961 28361 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "c1d212a9213d4d1f88747c422dd7c8c7"
format_stamp: "Formatted at 2026-11-18 03:06:38 on dist-test-slave-k60w"
I20261118 03:06:38.344973 28361 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:38.345602 28375 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.345752 28361 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.345808 28361 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "c1d212a9213d4d1f88747c422dd7c8c7"
format_stamp: "Formatted at 2026-11-18 03:06:38 on dist-test-slave-k60w"
I20261118 03:06:38.345860 28361 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:38.357751 28361 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:38.357981 28361 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:38.358080 28361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:38.358253 28361 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:38.358553 28361 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:38.358584 28361 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.358613 28361 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:38.358636 28361 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:38.363860 28361 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.132:36037
I20261118 03:06:38.363924 28488 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.132:36037 every 8 connection(s)
I20261118 03:06:38.364214 28361 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261118 03:06:38.368214 28489 heartbeater.cc:345] Connected to a master server at 127.24.137.190:38327
I20261118 03:06:38.368306 28489 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:38.368491 28489 heartbeater.cc:508] Master 127.24.137.190:38327 requested a full tablet report, sending...
I20261118 03:06:38.368834 27907 ts_manager.cc:210] Registered new tserver with Master: c1d212a9213d4d1f88747c422dd7c8c7 (127.24.137.132:36037)
I20261118 03:06:38.369189 27907 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.132:46123
I20261118 03:06:38.371750 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 28361
I20261118 03:06:38.371825 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20261118 03:06:38.372756 25126 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261118 03:06:38.378322 27907 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40566:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20261118 03:06:38.386111 28292 tablet_service.cc:1500] Processing CreateTablet for tablet ed3cb0f31b1b4f218370fb97609eb061 (DEFAULT_TABLE table=TestTable [id=17d8794fda3547b995a8f4bea6aa5321]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:38.386189 28030 tablet_service.cc:1500] Processing CreateTablet for tablet ed3cb0f31b1b4f218370fb97609eb061 (DEFAULT_TABLE table=TestTable [id=17d8794fda3547b995a8f4bea6aa5321]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:38.386214 28423 tablet_service.cc:1500] Processing CreateTablet for tablet ed3cb0f31b1b4f218370fb97609eb061 (DEFAULT_TABLE table=TestTable [id=17d8794fda3547b995a8f4bea6aa5321]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:38.386443 28292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3cb0f31b1b4f218370fb97609eb061. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:38.386441 28030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3cb0f31b1b4f218370fb97609eb061. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:38.386453 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3cb0f31b1b4f218370fb97609eb061. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:38.388612 28506 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Bootstrap starting.
I20261118 03:06:38.388964 28508 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Bootstrap starting.
I20261118 03:06:38.389214 28506 tablet_bootstrap.cc:654] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:38.389526 28508 tablet_bootstrap.cc:654] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:38.389526 28506 log.cc:826] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:38.389820 28508 log.cc:826] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:38.390108 28507 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: Bootstrap starting.
I20261118 03:06:38.390148 28506 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: No bootstrap required, opened a new log
I20261118 03:06:38.390206 28506 ts_tablet_manager.cc:1449] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20261118 03:06:38.390494 28508 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: No bootstrap required, opened a new log
I20261118 03:06:38.390560 28508 ts_tablet_manager.cc:1449] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20261118 03:06:38.390630 28507 tablet_bootstrap.cc:654] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:38.390841 28507 log.cc:826] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:38.391338 28507 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: No bootstrap required, opened a new log
I20261118 03:06:38.391392 28507 ts_tablet_manager.cc:1449] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20261118 03:06:38.391752 28506 raft_consensus.cc:360] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:38.391898 28506 raft_consensus.cc:386] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:38.391935 28506 raft_consensus.cc:748] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d212a9213d4d1f88747c422dd7c8c7, State: Initialized, Role: FOLLOWER
I20261118 03:06:38.391987 28508 raft_consensus.cc:360] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:38.392028 28506 consensus_queue.cc:261] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:38.392097 28508 raft_consensus.cc:386] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:38.392122 28508 raft_consensus.cc:748] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3d196e0e04469aaebe0da28a01598, State: Initialized, Role: FOLLOWER
I20261118 03:06:38.392261 28506 ts_tablet_manager.cc:1480] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 03:06:38.392230 28508 consensus_queue.cc:261] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:38.392381 28489 heartbeater.cc:500] Master 127.24.137.190:38327 was elected leader, sending a full tablet report...
I20261118 03:06:38.392450 28508 ts_tablet_manager.cc:1480] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 03:06:38.392627 28358 heartbeater.cc:500] Master 127.24.137.190:38327 was elected leader, sending a full tablet report...
I20261118 03:06:38.392767 28507 raft_consensus.cc:360] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:38.392885 28507 raft_consensus.cc:386] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:38.392910 28507 raft_consensus.cc:748] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb74cb32ea5464eabee31b59c7ab263, State: Initialized, Role: FOLLOWER
I20261118 03:06:38.392988 28507 consensus_queue.cc:261] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:38.393208 28507 ts_tablet_manager.cc:1480] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 03:06:38.393309 28096 heartbeater.cc:500] Master 127.24.137.190:38327 was elected leader, sending a full tablet report...
W20261118 03:06:38.425833 28097 tablet.cc:2441] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261118 03:06:38.453658 28359 tablet.cc:2441] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261118 03:06:38.464704 28490 tablet.cc:2441] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 03:06:38.584297 28512 raft_consensus.cc:499] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 03:06:38.584430 28512 raft_consensus.cc:521] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:38.584702 28512 leader_election.cc:290] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdb74cb32ea5464eabee31b59c7ab263 (127.24.137.129:46795), 8ef3d196e0e04469aaebe0da28a01598 (127.24.137.131:46181)
I20261118 03:06:38.587656 28312 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ed3cb0f31b1b4f218370fb97609eb061" candidate_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3d196e0e04469aaebe0da28a01598" is_pre_election: true
I20261118 03:06:38.587785 28050 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ed3cb0f31b1b4f218370fb97609eb061" candidate_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdb74cb32ea5464eabee31b59c7ab263" is_pre_election: true
I20261118 03:06:38.587815 28312 raft_consensus.cc:2490] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1d212a9213d4d1f88747c422dd7c8c7 in term 0.
I20261118 03:06:38.587894 28050 raft_consensus.cc:2490] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1d212a9213d4d1f88747c422dd7c8c7 in term 0.
I20261118 03:06:38.588058 28377 leader_election.cc:304] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3d196e0e04469aaebe0da28a01598, c1d212a9213d4d1f88747c422dd7c8c7; no voters:
I20261118 03:06:38.588229 28512 raft_consensus.cc:2826] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 03:06:38.588280 28512 raft_consensus.cc:499] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 03:06:38.588297 28512 raft_consensus.cc:3096] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:38.588902 28512 raft_consensus.cc:521] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:38.589022 28512 leader_election.cc:290] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [CANDIDATE]: Term 1 election: Requested vote from peers cdb74cb32ea5464eabee31b59c7ab263 (127.24.137.129:46795), 8ef3d196e0e04469aaebe0da28a01598 (127.24.137.131:46181)
I20261118 03:06:38.589154 28050 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ed3cb0f31b1b4f218370fb97609eb061" candidate_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdb74cb32ea5464eabee31b59c7ab263"
I20261118 03:06:38.589185 28312 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ed3cb0f31b1b4f218370fb97609eb061" candidate_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3d196e0e04469aaebe0da28a01598"
I20261118 03:06:38.589242 28312 raft_consensus.cc:3096] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:38.589238 28050 raft_consensus.cc:3096] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:38.590045 28050 raft_consensus.cc:2490] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1d212a9213d4d1f88747c422dd7c8c7 in term 1.
I20261118 03:06:38.590214 28378 leader_election.cc:304] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1d212a9213d4d1f88747c422dd7c8c7, cdb74cb32ea5464eabee31b59c7ab263; no voters:
I20261118 03:06:38.590313 28512 raft_consensus.cc:2826] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:38.590361 28312 raft_consensus.cc:2490] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1d212a9213d4d1f88747c422dd7c8c7 in term 1.
I20261118 03:06:38.590469 28512 raft_consensus.cc:705] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 1 LEADER]: Becoming Leader. State: Replica: c1d212a9213d4d1f88747c422dd7c8c7, State: Running, Role: LEADER
I20261118 03:06:38.590569 28512 consensus_queue.cc:238] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:38.591344 27908 catalog_manager.cc:5840] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1d212a9213d4d1f88747c422dd7c8c7 (127.24.137.132). New cstate: current_term: 1 leader_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } health_report { overall_health: UNKNOWN } } }
I20261118 03:06:38.591495 28516 consensus_queue.cc:1049] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 03:06:38.591614 28518 consensus_queue.cc:1049] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 03:06:38.608209 25126 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261118 03:06:38.608652 25126 ts_itest-base.cc:246] Waiting for 1 tablets on tserver cdb74cb32ea5464eabee31b59c7ab263 to finish bootstrapping
I20261118 03:06:38.612097 25126 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 8ef3d196e0e04469aaebe0da28a01598 to finish bootstrapping
I20261118 03:06:38.614913 25126 ts_itest-base.cc:246] Waiting for 1 tablets on tserver c1d212a9213d4d1f88747c422dd7c8c7 to finish bootstrapping
I20261118 03:06:38.617921 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 28230
I20261118 03:06:38.629793 25126 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20261118 03:06:38.630033 25126 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
W20261118 03:06:38.636178 28377 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.137.131:46181: connect: Connection refused (error 111)
W20261118 03:06:38.636785 28377 consensus_peers.cc:597] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 -> Peer 8ef3d196e0e04469aaebe0da28a01598 (127.24.137.131:46181): Couldn't send request to peer 8ef3d196e0e04469aaebe0da28a01598. Status: Network error: Client connection negotiation failed: client connection to 127.24.137.131:46181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 03:06:38.665328 28490 maintenance_manager.cc:419] P c1d212a9213d4d1f88747c422dd7c8c7: Scheduling FlushMRSOp(ed3cb0f31b1b4f218370fb97609eb061): 3681860 bytes log retention, and flush 5079305 bytes memory
I20261118 03:06:38.671947 28380 maintenance_manager.cc:643] P c1d212a9213d4d1f88747c422dd7c8c7: FlushMRSOp(ed3cb0f31b1b4f218370fb97609eb061) complete. Timing: real 0.006s user 0.002s sys 0.002s Metrics: {"bytes_written":135937,"cfile_init":1,"compiler_manager_pool.queue_time_us":130,"dirs.queue_time_us":53,"dirs.run_cpu_time_us":151,"dirs.run_wall_time_us":1024,"drs_written":1,"lbm_read_time_us":11,"lbm_reads_lt_1ms":4,"lbm_write_time_us":190,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":32,"thread_start_us":97,"threads_started":1}
I20261118 03:06:38.672160 28490 maintenance_manager.cc:419] P c1d212a9213d4d1f88747c422dd7c8c7: Scheduling LogGCOp(ed3cb0f31b1b4f218370fb97609eb061): free 2761328 bytes of WAL
I20261118 03:06:38.672283 28380 log_reader.cc:385] T ed3cb0f31b1b4f218370fb97609eb061: removed 3 log segments from log reader
I20261118 03:06:38.672329 28380 log.cc:1079] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/wals/ed3cb0f31b1b4f218370fb97609eb061/wal-000000001 (ops 1-8)
I20261118 03:06:38.672411 28380 log.cc:1079] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/wals/ed3cb0f31b1b4f218370fb97609eb061/wal-000000002 (ops 9-15)
I20261118 03:06:38.672429 28380 log.cc:1079] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/wals/ed3cb0f31b1b4f218370fb97609eb061/wal-000000003 (ops 16-22)
I20261118 03:06:38.673337 28380 maintenance_manager.cc:643] P c1d212a9213d4d1f88747c422dd7c8c7: LogGCOp(ed3cb0f31b1b4f218370fb97609eb061) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261118 03:06:38.673460 28490 maintenance_manager.cc:419] P c1d212a9213d4d1f88747c422dd7c8c7: Scheduling UndoDeltaBlockGCOp(ed3cb0f31b1b4f218370fb97609eb061): 488 bytes on disk
I20261118 03:06:38.673645 28380 maintenance_manager.cc:643] P c1d212a9213d4d1f88747c422dd7c8c7: UndoDeltaBlockGCOp(ed3cb0f31b1b4f218370fb97609eb061) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":19,"lbm_reads_lt_1ms":4}
I20261118 03:06:38.675247 25126 raft_consensus-itest-base.cc:274] Waiting for log GC on c1d212a9213d4d1f88747c422dd7c8c7
I20261118 03:06:38.675407 25126 raft_consensus-itest-base.cc:279] Log GC complete on c1d212a9213d4d1f88747c422dd7c8c7
I20261118 03:06:38.726624 28097 maintenance_manager.cc:419] P cdb74cb32ea5464eabee31b59c7ab263: Scheduling FlushMRSOp(ed3cb0f31b1b4f218370fb97609eb061): 4602219 bytes log retention, and flush 6119689 bytes memory
I20261118 03:06:38.732846 27987 maintenance_manager.cc:643] P cdb74cb32ea5464eabee31b59c7ab263: FlushMRSOp(ed3cb0f31b1b4f218370fb97609eb061) complete. Timing: real 0.006s user 0.005s sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":166,"dirs.queue_time_us":186,"dirs.run_cpu_time_us":168,"dirs.run_wall_time_us":763,"drs_written":1,"lbm_read_time_us":20,"lbm_reads_lt_1ms":4,"lbm_write_time_us":291,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":38,"thread_start_us":231,"threads_started":2}
I20261118 03:06:38.733055 28097 maintenance_manager.cc:419] P cdb74cb32ea5464eabee31b59c7ab263: Scheduling LogGCOp(ed3cb0f31b1b4f218370fb97609eb061): free 3155800 bytes of WAL
I20261118 03:06:38.733155 27987 log_reader.cc:385] T ed3cb0f31b1b4f218370fb97609eb061: removed 4 log segments from log reader
I20261118 03:06:38.733187 27987 log.cc:1079] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal/wals/ed3cb0f31b1b4f218370fb97609eb061/wal-000000001 (ops 1-7)
I20261118 03:06:38.733209 27987 log.cc:1079] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal/wals/ed3cb0f31b1b4f218370fb97609eb061/wal-000000002 (ops 8-14)
I20261118 03:06:38.733270 27987 log.cc:1079] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal/wals/ed3cb0f31b1b4f218370fb97609eb061/wal-000000003 (ops 15-20)
I20261118 03:06:38.733297 27987 log.cc:1079] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal/wals/ed3cb0f31b1b4f218370fb97609eb061/wal-000000004 (ops 21-25)
I20261118 03:06:38.734273 27987 maintenance_manager.cc:643] P cdb74cb32ea5464eabee31b59c7ab263: LogGCOp(ed3cb0f31b1b4f218370fb97609eb061) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261118 03:06:38.734387 28097 maintenance_manager.cc:419] P cdb74cb32ea5464eabee31b59c7ab263: Scheduling UndoDeltaBlockGCOp(ed3cb0f31b1b4f218370fb97609eb061): 558 bytes on disk
I20261118 03:06:38.734573 27987 maintenance_manager.cc:643] P cdb74cb32ea5464eabee31b59c7ab263: UndoDeltaBlockGCOp(ed3cb0f31b1b4f218370fb97609eb061) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":21,"lbm_reads_lt_1ms":4}
I20261118 03:06:39.146221 28227 heartbeater.cc:500] Master 127.24.137.190:38327 was elected leader, sending a full tablet report...
I20261118 03:06:39.257220 28541 consensus_queue.cc:754] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [LEADER]: The logs necessary to catch up peer 8ef3d196e0e04469aaebe0da28a01598 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 4..34: Segment 1 which contained index 4 has been GCed)
I20261118 03:06:39.257295 28541 consensus_queue.cc:580] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [LEADER]: The logs necessary to catch up peer 8ef3d196e0e04469aaebe0da28a01598 have been garbage collected. The replica will never be able to catch up
I20261118 03:06:39.258378 28442 consensus_queue.cc:238] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 1.39, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } }
I20261118 03:06:39.258860 28050 raft_consensus.cc:1285] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [term 1 FOLLOWER]: Refusing update from remote peer c1d212a9213d4d1f88747c422dd7c8c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 40. (index mismatch)
I20261118 03:06:39.259102 28548 consensus_queue.cc:1049] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20261118 03:06:39.259764 28541 raft_consensus.cc:2983] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 1 LEADER]: Committing config change with OpId 1.40: config changed from index -1 to 40, VOTER 8ef3d196e0e04469aaebe0da28a01598 (127.24.137.131) evicted. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } }
I20261118 03:06:39.259886 28050 raft_consensus.cc:2983] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [term 1 FOLLOWER]: Committing config change with OpId 1.40: config changed from index -1 to 40, VOTER 8ef3d196e0e04469aaebe0da28a01598 (127.24.137.131) evicted. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } }
I20261118 03:06:39.260867 27892 catalog_manager.cc:5247] ChangeConfig:REMOVE_PEER RPC for tablet ed3cb0f31b1b4f218370fb97609eb061 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20261118 03:06:39.261082 27907 catalog_manager.cc:5840] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 reported cstate change: config changed from index -1 to 40, VOTER 8ef3d196e0e04469aaebe0da28a01598 (127.24.137.131) evicted. New cstate: current_term: 1 leader_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } }
W20261118 03:06:39.262252 27893 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.137.131:46181: connect: Connection refused (error 111)
I20261118 03:06:39.262236 28442 consensus_queue.cc:238] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 1.40, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "dd22ed538cf0402f9542416dc9d2ca64" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 38049 } attrs { promote: true } }
W20261118 03:06:39.262811 27893 catalog_manager.cc:4792] TS 8ef3d196e0e04469aaebe0da28a01598 (127.24.137.131:46181): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet ed3cb0f31b1b4f218370fb97609eb061: Network error: Client connection negotiation failed: client connection to 127.24.137.131:46181: connect: Connection refused (error 111)
I20261118 03:06:39.262905 28050 raft_consensus.cc:1285] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [term 1 FOLLOWER]: Refusing update from remote peer c1d212a9213d4d1f88747c422dd7c8c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 40. Preceding OpId from leader: term: 1 index: 41. (index mismatch)
I20261118 03:06:39.263113 28550 consensus_queue.cc:1049] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20261118 03:06:39.263911 28541 raft_consensus.cc:2983] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 [term 1 LEADER]: Committing config change with OpId 1.41: config changed from index 40 to 41, NON_VOTER dd22ed538cf0402f9542416dc9d2ca64 (127.24.137.130) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "dd22ed538cf0402f9542416dc9d2ca64" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 38049 } attrs { promote: true } } }
I20261118 03:06:39.264089 28050 raft_consensus.cc:2983] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 [term 1 FOLLOWER]: Committing config change with OpId 1.41: config changed from index 40 to 41, NON_VOTER dd22ed538cf0402f9542416dc9d2ca64 (127.24.137.130) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "dd22ed538cf0402f9542416dc9d2ca64" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 38049 } attrs { promote: true } } }
I20261118 03:06:39.264969 27892 catalog_manager.cc:5247] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ed3cb0f31b1b4f218370fb97609eb061 with cas_config_opid_index 40: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20261118 03:06:39.265131 27908 catalog_manager.cc:5840] T ed3cb0f31b1b4f218370fb97609eb061 P cdb74cb32ea5464eabee31b59c7ab263 reported cstate change: config changed from index 40 to 41, NON_VOTER dd22ed538cf0402f9542416dc9d2ca64 (127.24.137.130) added. New cstate: current_term: 1 leader_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" committed_config { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "dd22ed538cf0402f9542416dc9d2ca64" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 38049 } attrs { promote: true } } }
W20261118 03:06:39.267113 28379 consensus_peers.cc:597] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7 -> Peer dd22ed538cf0402f9542416dc9d2ca64 (127.24.137.130:38049): Couldn't send request to peer dd22ed538cf0402f9542416dc9d2ca64. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ed3cb0f31b1b4f218370fb97609eb061. This is attempt 1: this message will repeat every 5th retry.
I20261118 03:06:39.885957 28558 ts_tablet_manager.cc:979] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: Initiating tablet copy from peer c1d212a9213d4d1f88747c422dd7c8c7 (127.24.137.132:36037)
I20261118 03:06:39.886142 28558 tablet_copy_client.cc:323] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: tablet copy: Beginning tablet copy session from remote peer at address 127.24.137.132:36037
I20261118 03:06:39.888684 28463 tablet_copy_service.cc:140] P c1d212a9213d4d1f88747c422dd7c8c7: Received BeginTabletCopySession request for tablet ed3cb0f31b1b4f218370fb97609eb061 from peer dd22ed538cf0402f9542416dc9d2ca64 ({username='slave'} at 127.24.137.130:47989)
I20261118 03:06:39.888761 28463 tablet_copy_service.cc:161] P c1d212a9213d4d1f88747c422dd7c8c7: Beginning new tablet copy session on tablet ed3cb0f31b1b4f218370fb97609eb061 from peer dd22ed538cf0402f9542416dc9d2ca64 at {username='slave'} at 127.24.137.130:47989: session id = dd22ed538cf0402f9542416dc9d2ca64-ed3cb0f31b1b4f218370fb97609eb061
I20261118 03:06:39.889353 28463 tablet_copy_source_session.cc:215] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Tablet Copy: opened 5 blocks and 3 log segments
I20261118 03:06:39.890110 28558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3cb0f31b1b4f218370fb97609eb061. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:39.891865 28558 tablet_copy_client.cc:806] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: tablet copy: Starting download of 5 data blocks...
I20261118 03:06:39.894444 28558 tablet_copy_client.cc:670] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: tablet copy: Starting download of 3 WAL segments...
I20261118 03:06:39.900532 28558 tablet_copy_client.cc:538] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261118 03:06:39.901333 28558 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: Bootstrap starting.
I20261118 03:06:39.903036 28558 log.cc:826] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:39.905659 28558 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: Bootstrap replayed 1/3 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=7} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 0 replicates
I20261118 03:06:39.907173 28558 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: Bootstrap replayed 2/3 log segments. Stats: ops{read=14 overwritten=0 applied=11 ignored=11} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 3 replicates
I20261118 03:06:39.908545 28558 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: Bootstrap replayed 3/3 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=11} inserts{seen=6 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 0 replicates
I20261118 03:06:39.908775 28558 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: Bootstrap complete.
I20261118 03:06:39.909212 28558 ts_tablet_manager.cc:1449] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20261118 03:06:39.910017 28558 raft_consensus.cc:360] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "dd22ed538cf0402f9542416dc9d2ca64" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 38049 } attrs { promote: true } }
I20261118 03:06:39.910144 28558 raft_consensus.cc:748] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd22ed538cf0402f9542416dc9d2ca64, State: Initialized, Role: LEARNER
I20261118 03:06:39.910197 28558 consensus_queue.cc:261] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 41, Last appended: 1.41, Last appended by leader: 41, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "dd22ed538cf0402f9542416dc9d2ca64" member_type: NON_VOTER last_known_addr { host: "127.24.137.130" port: 38049 } attrs { promote: true } }
I20261118 03:06:39.910421 28558 ts_tablet_manager.cc:1480] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20261118 03:06:39.910825 28463 tablet_copy_service.cc:342] P c1d212a9213d4d1f88747c422dd7c8c7: Request end of tablet copy session dd22ed538cf0402f9542416dc9d2ca64-ed3cb0f31b1b4f218370fb97609eb061 received from {username='slave'} at 127.24.137.130:47989
I20261118 03:06:39.910894 28463 tablet_copy_service.cc:434] P c1d212a9213d4d1f88747c422dd7c8c7: ending tablet copy session dd22ed538cf0402f9542416dc9d2ca64-ed3cb0f31b1b4f218370fb97609eb061 on tablet ed3cb0f31b1b4f218370fb97609eb061 with peer dd22ed538cf0402f9542416dc9d2ca64
I20261118 03:06:39.943301 28228 maintenance_manager.cc:419] P dd22ed538cf0402f9542416dc9d2ca64: Scheduling UndoDeltaBlockGCOp(ed3cb0f31b1b4f218370fb97609eb061): 488 bytes on disk
I20261118 03:06:39.943657 28118 maintenance_manager.cc:643] P dd22ed538cf0402f9542416dc9d2ca64: UndoDeltaBlockGCOp(ed3cb0f31b1b4f218370fb97609eb061) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":18,"lbm_reads_lt_1ms":4}
I20261118 03:06:40.425313 28181 raft_consensus.cc:1227] T ed3cb0f31b1b4f218370fb97609eb061 P dd22ed538cf0402f9542416dc9d2ca64 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.40->[1.41-1.41] Dedup: 1.41->[]
W20261118 03:06:40.468837 27893 catalog_manager.cc:4792] TS 8ef3d196e0e04469aaebe0da28a01598 (127.24.137.131:46181): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet ed3cb0f31b1b4f218370fb97609eb061: Network error: Client connection negotiation failed: client connection to 127.24.137.131:46181: connect: Connection refused (error 111)
I20261118 03:06:40.476822 28490 maintenance_manager.cc:419] P c1d212a9213d4d1f88747c422dd7c8c7: Scheduling LogGCOp(ed3cb0f31b1b4f218370fb97609eb061): free 920532 bytes of WAL
I20261118 03:06:40.476975 28380 log_reader.cc:385] T ed3cb0f31b1b4f218370fb97609eb061: removed 1 log segments from log reader
I20261118 03:06:40.477007 28380 log.cc:1079] T ed3cb0f31b1b4f218370fb97609eb061 P c1d212a9213d4d1f88747c422dd7c8c7: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/wals/ed3cb0f31b1b4f218370fb97609eb061/wal-000000004 (ops 23-29)
I20261118 03:06:40.477243 28380 maintenance_manager.cc:643] P c1d212a9213d4d1f88747c422dd7c8c7: LogGCOp(ed3cb0f31b1b4f218370fb97609eb061) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20261118 03:06:40.676072 25126 raft_consensus-itest-base.cc:295] Servers converged with original term 1
I20261118 03:06:40.676138 25126 raft_consensus-itest-base.cc:303] Restarting 8ef3d196e0e04469aaebe0da28a01598
I20261118 03:06:40.676294 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.131:46181
--local_ip_for_outbound_sockets=127.24.137.131
--tserver_master_addrs=127.24.137.190:38327
--webserver_port=34063
--webserver_interface=127.24.137.131
--builtin_ntp_servers=127.24.137.148:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261118 03:06:40.752112 28571 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:40.752297 28571 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:40.752319 28571 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:40.752341 28571 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:40.753960 28571 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:40.754009 28571 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:40.754030 28571 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:40.754060 28571 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:40.754087 28571 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.131
I20261118 03:06:40.755743 28571 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.131:46181
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.137.131
--webserver_port=34063
--tserver_master_addrs=127.24.137.190:38327
--never_fsync=true
--heap_profile_path=/tmp/kudu.28571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:40.755950 28571 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:40.756109 28571 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:40.758338 28579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:40.758371 28577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:40.758368 28576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:40.758585 28571 server_base.cc:1061] running on GCE node
I20261118 03:06:40.758757 28571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:40.758944 28571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:40.760075 28571 hybrid_clock.cc:648] HybridClock initialized: now 1794971200760062 us; error 27 us; skew 500 ppm
I20261118 03:06:40.761058 28571 webserver.cc:577] Webserver started at http://127.24.137.131:34063/ using document root <none> and password file <none>
I20261118 03:06:40.761221 28571 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:40.761277 28571 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:40.762374 28571 fs_manager.cc:714] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:40.762907 28585 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:40.763041 28571 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:40.763098 28571 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "8ef3d196e0e04469aaebe0da28a01598"
format_stamp: "Formatted at 2026-11-18 03:06:38 on dist-test-slave-k60w"
I20261118 03:06:40.763365 28571 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:40.772725 28571 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:40.772948 28571 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:40.773054 28571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:40.773236 28571 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:40.773672 28592 ts_tablet_manager.cc:588] Loading tablet metadata (0/1 complete)
I20261118 03:06:40.774508 28571 ts_tablet_manager.cc:631] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20261118 03:06:40.774564 28571 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20261118 03:06:40.774597 28571 ts_tablet_manager.cc:646] Registering tablets (0/1 complete)
I20261118 03:06:40.775163 28571 ts_tablet_manager.cc:662] Registered 1 tablets
I20261118 03:06:40.775197 28571 ts_tablet_manager.cc:641] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20261118 03:06:40.775271 28592 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Bootstrap starting.
I20261118 03:06:40.778163 28592 log.cc:826] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:40.779083 28592 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 03:06:40.779350 28592 tablet_bootstrap.cc:492] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Bootstrap complete.
I20261118 03:06:40.779573 28592 ts_tablet_manager.cc:1449] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20261118 03:06:40.780802 28592 raft_consensus.cc:360] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:40.780990 28592 raft_consensus.cc:748] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3d196e0e04469aaebe0da28a01598, State: Initialized, Role: FOLLOWER
I20261118 03:06:40.781101 28592 consensus_queue.cc:261] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d212a9213d4d1f88747c422dd7c8c7" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36037 } } peers { permanent_uuid: "cdb74cb32ea5464eabee31b59c7ab263" member_type: VOTER last_known_addr { host: "127.24.137.129" port: 46795 } } peers { permanent_uuid: "8ef3d196e0e04469aaebe0da28a01598" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 46181 } }
I20261118 03:06:40.781348 28592 ts_tablet_manager.cc:1480] T ed3cb0f31b1b4f218370fb97609eb061 P 8ef3d196e0e04469aaebe0da28a01598: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20261118 03:06:40.781597 28571 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.131:46181
I20261118 03:06:40.781627 28701 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.131:46181 every 8 connection(s)
I20261118 03:06:40.781908 28571 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehind.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261118 03:06:40.785600 28702 heartbeater.cc:345] Connected to a master server at 127.24.137.190:38327
I20261118 03:06:40.785696 28702 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:40.785844 28702 heartbeater.cc:508] Master 127.24.137.190:38327 requested a full tablet report, sending...
I20261118 03:06:40.786324 27907 ts_manager.cc:210] Re-registered known tserver with Master: 8ef3d196e0e04469aaebe0da28a01598 (127.24.137.131:46181)
I20261118 03:06:40.786688 27907 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.131:55319
I20261118 03:06:40.790544 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 28571
I20261118 03:06:40.791989 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 27968
I20261118 03:06:40.797058 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 28099
I20261118 03:06:40.801317 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 28571
I20261118 03:06:40.805341 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 28361
I20261118 03:06:40.810276 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 27877
2026-11-18T03:06:40Z chronyd exiting
[ OK ] SlowTabletBootstrapTest.TestFallBehind (3100 ms)
[ RUN ] SlowTabletBootstrapTest.TestFallBehindSlowBootstrap
I20261118 03:06:40.831568 25126 ts_itest-base.cc:115] Starting cluster with:
I20261118 03:06:40.831617 25126 ts_itest-base.cc:116] --------------
I20261118 03:06:40.831629 25126 ts_itest-base.cc:117] 4 tablet servers
I20261118 03:06:40.831637 25126 ts_itest-base.cc:118] 3 replicas per TS
I20261118 03:06:40.831646 25126 ts_itest-base.cc:119] --------------
2026-11-18T03:06:40Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-18T03:06:40Z Disabled control of system clock
I20261118 03:06:40.838486 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:37643
--webserver_interface=127.24.137.190
--webserver_port=0
--builtin_ntp_servers=127.24.137.148:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.137.190:37643 with env {}
W20261118 03:06:40.912856 28713 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:40.913022 28713 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:40.913038 28713 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:40.914641 28713 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 03:06:40.914700 28713 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:40.914713 28713 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 03:06:40.914724 28713 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 03:06:40.916332 28713 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.137.190:37643
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.137.190:37643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.24.137.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.28713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:40.916536 28713 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:40.916704 28713 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:40.918700 28718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:40.918711 28721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:40.918696 28719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:40.919070 28713 server_base.cc:1061] running on GCE node
I20261118 03:06:40.919250 28713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:40.919441 28713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:40.920563 28713 hybrid_clock.cc:648] HybridClock initialized: now 1794971200920553 us; error 24 us; skew 500 ppm
I20261118 03:06:40.921435 28713 webserver.cc:577] Webserver started at http://127.24.137.190:39037/ using document root <none> and password file <none>
I20261118 03:06:40.921635 28713 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:40.921675 28713 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:40.921761 28713 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:40.922549 28713 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "e1456ac8f891447fab4a2b81b7bbd88d"
format_stamp: "Formatted at 2026-11-18 03:06:40 on dist-test-slave-k60w"
I20261118 03:06:40.922834 28713 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "e1456ac8f891447fab4a2b81b7bbd88d"
format_stamp: "Formatted at 2026-11-18 03:06:40 on dist-test-slave-k60w"
I20261118 03:06:40.923842 28713 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261118 03:06:40.924423 28727 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:40.924576 28713 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:40.924633 28713 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
uuid: "e1456ac8f891447fab4a2b81b7bbd88d"
format_stamp: "Formatted at 2026-11-18 03:06:40 on dist-test-slave-k60w"
I20261118 03:06:40.924685 28713 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:40.933568 28713 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:40.933804 28713 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:40.933909 28713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:40.937475 28713 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.190:37643
I20261118 03:06:40.937505 28779 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.190:37643 every 8 connection(s)
I20261118 03:06:40.937832 28713 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20261118 03:06:40.938364 28780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:40.940400 28780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d: Bootstrap starting.
I20261118 03:06:40.940936 28780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:40.941110 28780 log.cc:826] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:40.941634 28780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d: No bootstrap required, opened a new log
I20261118 03:06:40.941941 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 28713
I20261118 03:06:40.942034 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/master-0/wal/instance
I20261118 03:06:40.943300 28780 raft_consensus.cc:360] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1456ac8f891447fab4a2b81b7bbd88d" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 37643 } }
I20261118 03:06:40.943454 28780 raft_consensus.cc:386] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:40.943483 28780 raft_consensus.cc:748] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1456ac8f891447fab4a2b81b7bbd88d, State: Initialized, Role: FOLLOWER
I20261118 03:06:40.943578 28780 consensus_queue.cc:261] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1456ac8f891447fab4a2b81b7bbd88d" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 37643 } }
I20261118 03:06:40.943638 28780 raft_consensus.cc:400] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20261118 03:06:40.943665 28780 raft_consensus.cc:499] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20261118 03:06:40.943703 28780 raft_consensus.cc:3096] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:40.944240 28780 raft_consensus.cc:521] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1456ac8f891447fab4a2b81b7bbd88d" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 37643 } }
I20261118 03:06:40.944350 28780 leader_election.cc:304] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1456ac8f891447fab4a2b81b7bbd88d; no voters:
I20261118 03:06:40.944489 28780 leader_election.cc:290] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [CANDIDATE]: Term 1 election: Requested vote from peers
I20261118 03:06:40.944521 28785 raft_consensus.cc:2826] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:40.944609 28785 raft_consensus.cc:705] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [term 1 LEADER]: Becoming Leader. State: Replica: e1456ac8f891447fab4a2b81b7bbd88d, State: Running, Role: LEADER
I20261118 03:06:40.944710 28780 sys_catalog.cc:565] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [sys.catalog]: configured and running, proceeding with master startup.
I20261118 03:06:40.944763 28785 consensus_queue.cc:238] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1456ac8f891447fab4a2b81b7bbd88d" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 37643 } }
I20261118 03:06:40.945091 28785 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1456ac8f891447fab4a2b81b7bbd88d. Latest consensus state: current_term: 1 leader_uuid: "e1456ac8f891447fab4a2b81b7bbd88d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1456ac8f891447fab4a2b81b7bbd88d" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 37643 } } }
I20261118 03:06:40.945324 28785 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [sys.catalog]: This master's current role is: LEADER
I20261118 03:06:40.945261 28787 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1456ac8f891447fab4a2b81b7bbd88d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1456ac8f891447fab4a2b81b7bbd88d" member_type: VOTER last_known_addr { host: "127.24.137.190" port: 37643 } } }
I20261118 03:06:40.945526 28787 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d [sys.catalog]: This master's current role is: LEADER
I20261118 03:06:40.945910 28794 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 03:06:40.946383 28794 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 03:06:40.947711 28794 catalog_manager.cc:1398] Generated new cluster ID: bb10d4831ddd4a4497353e20060697a4
I20261118 03:06:40.947757 28794 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 03:06:40.965245 28794 catalog_manager.cc:1421] Generated new certificate authority record
I20261118 03:06:40.965940 28794 catalog_manager.cc:1555] Loading token signing keys...
I20261118 03:06:40.973672 28794 catalog_manager.cc:6221] T 00000000000000000000000000000000 P e1456ac8f891447fab4a2b81b7bbd88d: Generated new TSK 0
I20261118 03:06:40.973841 28794 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 03:06:40.978722 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.129:0
--local_ip_for_outbound_sockets=127.24.137.129
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:37643
--builtin_ntp_servers=127.24.137.148:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261118 03:06:41.057222 28804 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:41.057389 28804 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:41.057415 28804 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:41.057439 28804 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:41.059041 28804 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:41.059093 28804 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:41.059110 28804 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:41.059132 28804 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:41.059146 28804 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.129
I20261118 03:06:41.060766 28804 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.24.137.129
--webserver_port=0
--tserver_master_addrs=127.24.137.190:37643
--never_fsync=true
--heap_profile_path=/tmp/kudu.28804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:41.060949 28804 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:41.061136 28804 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:41.063529 28810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:41.063535 28809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:41.063673 28804 server_base.cc:1061] running on GCE node
W20261118 03:06:41.063742 28812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:41.063921 28804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:41.064105 28804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:41.065227 28804 hybrid_clock.cc:648] HybridClock initialized: now 1794971201065218 us; error 26 us; skew 500 ppm
I20261118 03:06:41.066288 28804 webserver.cc:577] Webserver started at http://127.24.137.129:33153/ using document root <none> and password file <none>
I20261118 03:06:41.066489 28804 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:41.066525 28804 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:41.066603 28804 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:41.067469 28804 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "65f03f692fe54f7abc568ed3bc7caf9e"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.067765 28804 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "65f03f692fe54f7abc568ed3bc7caf9e"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.068794 28804 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261118 03:06:41.069465 28818 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.069615 28804 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:41.069674 28804 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "65f03f692fe54f7abc568ed3bc7caf9e"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.069727 28804 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:41.087411 28804 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:41.087606 28804 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:41.087707 28804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:41.087900 28804 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:41.088243 28804 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:41.088281 28804 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.088320 28804 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:41.088343 28804 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.094786 28804 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.129:43317
I20261118 03:06:41.094844 28931 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.129:43317 every 8 connection(s)
I20261118 03:06:41.095127 28804 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20261118 03:06:41.099381 28932 heartbeater.cc:345] Connected to a master server at 127.24.137.190:37643
I20261118 03:06:41.099478 28932 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:41.099628 28932 heartbeater.cc:508] Master 127.24.137.190:37643 requested a full tablet report, sending...
I20261118 03:06:41.100003 28744 ts_manager.cc:210] Registered new tserver with Master: 65f03f692fe54f7abc568ed3bc7caf9e (127.24.137.129:43317)
I20261118 03:06:41.100620 28744 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.129:37237
I20261118 03:06:41.103580 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 28804
I20261118 03:06:41.103667 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20261118 03:06:41.104579 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.130:0
--local_ip_for_outbound_sockets=127.24.137.130
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:37643
--builtin_ntp_servers=127.24.137.148:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261118 03:06:41.178658 28935 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:41.178807 28935 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:41.178823 28935 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:41.178838 28935 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:41.180335 28935 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:41.180379 28935 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:41.180394 28935 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:41.180414 28935 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:41.180425 28935 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.130
I20261118 03:06:41.182003 28935 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.137.130
--webserver_port=0
--tserver_master_addrs=127.24.137.190:37643
--never_fsync=true
--heap_profile_path=/tmp/kudu.28935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:41.182163 28935 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:41.182293 28935 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:41.184615 28943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:41.184646 28941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:41.184762 28935 server_base.cc:1061] running on GCE node
W20261118 03:06:41.184860 28940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:41.185025 28935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:41.185182 28935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:41.186296 28935 hybrid_clock.cc:648] HybridClock initialized: now 1794971201186283 us; error 33 us; skew 500 ppm
I20261118 03:06:41.187300 28935 webserver.cc:577] Webserver started at http://127.24.137.130:42701/ using document root <none> and password file <none>
I20261118 03:06:41.187474 28935 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:41.187511 28935 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:41.187597 28935 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:41.188386 28935 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "9007147e0bff4e8d877a637c02284b26"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.188663 28935 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "9007147e0bff4e8d877a637c02284b26"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.189678 28935 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20261118 03:06:41.190313 28949 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.190438 28935 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.190490 28935 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "9007147e0bff4e8d877a637c02284b26"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.190539 28935 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:41.222164 28935 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:41.222391 28935 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:41.222483 28935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:41.222661 28935 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:41.222959 28935 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:41.222990 28935 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.223012 28935 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:41.223034 28935 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.228410 28935 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.130:43619
I20261118 03:06:41.228453 29062 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.130:43619 every 8 connection(s)
I20261118 03:06:41.228770 28935 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20261118 03:06:41.232841 29063 heartbeater.cc:345] Connected to a master server at 127.24.137.190:37643
I20261118 03:06:41.232932 29063 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:41.233105 29063 heartbeater.cc:508] Master 127.24.137.190:37643 requested a full tablet report, sending...
I20261118 03:06:41.233500 28744 ts_manager.cc:210] Registered new tserver with Master: 9007147e0bff4e8d877a637c02284b26 (127.24.137.130:43619)
I20261118 03:06:41.233886 28744 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.130:40381
I20261118 03:06:41.238855 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 28935
I20261118 03:06:41.238942 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20261118 03:06:41.239959 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.131:0
--local_ip_for_outbound_sockets=127.24.137.131
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:37643
--builtin_ntp_servers=127.24.137.148:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261118 03:06:41.313629 29066 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:41.313786 29066 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:41.313808 29066 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:41.313830 29066 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:41.315359 29066 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:41.315407 29066 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:41.315426 29066 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:41.315452 29066 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:41.315469 29066 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.131
I20261118 03:06:41.317050 29066 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.137.131
--webserver_port=0
--tserver_master_addrs=127.24.137.190:37643
--never_fsync=true
--heap_profile_path=/tmp/kudu.29066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:41.317245 29066 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:41.317410 29066 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:41.319574 29072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:41.319584 29074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:41.319635 29071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:41.319839 29066 server_base.cc:1061] running on GCE node
I20261118 03:06:41.319968 29066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:41.320128 29066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:41.321233 29066 hybrid_clock.cc:648] HybridClock initialized: now 1794971201321217 us; error 33 us; skew 500 ppm
I20261118 03:06:41.322276 29066 webserver.cc:577] Webserver started at http://127.24.137.131:45745/ using document root <none> and password file <none>
I20261118 03:06:41.322438 29066 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:41.322475 29066 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:41.322556 29066 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:41.323323 29066 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.323608 29066 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.324683 29066 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20261118 03:06:41.325313 29080 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.325480 29066 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20261118 03:06:41.325573 29066 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.325627 29066 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:41.361757 29066 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:41.361989 29066 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:41.362093 29066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:41.362274 29066 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:41.362602 29066 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:41.362633 29066 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.362663 29066 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:41.362679 29066 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.368134 29066 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.131:37177
I20261118 03:06:41.368175 29193 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.131:37177 every 8 connection(s)
I20261118 03:06:41.368482 29066 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20261118 03:06:41.372566 29194 heartbeater.cc:345] Connected to a master server at 127.24.137.190:37643
I20261118 03:06:41.372674 29194 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:41.372859 29194 heartbeater.cc:508] Master 127.24.137.190:37643 requested a full tablet report, sending...
I20261118 03:06:41.373194 28744 ts_manager.cc:210] Registered new tserver with Master: dc5e1d4991ee44b1a9bb3b6ca7c0e034 (127.24.137.131:37177)
I20261118 03:06:41.373596 28744 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.131:36685
I20261118 03:06:41.374066 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 29066
I20261118 03:06:41.374154 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20261118 03:06:41.375094 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.132:0
--local_ip_for_outbound_sockets=127.24.137.132
--webserver_interface=127.24.137.132
--webserver_port=0
--tserver_master_addrs=127.24.137.190:37643
--builtin_ntp_servers=127.24.137.148:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20261118 03:06:41.449662 29197 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:41.449810 29197 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:41.449824 29197 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:41.449841 29197 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:41.451414 29197 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:41.451463 29197 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:41.451478 29197 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:41.451500 29197 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:41.451526 29197 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.132
I20261118 03:06:41.453118 29197 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.137.132
--webserver_port=0
--tserver_master_addrs=127.24.137.190:37643
--never_fsync=true
--heap_profile_path=/tmp/kudu.29197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:41.453307 29197 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:41.453475 29197 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:41.455673 29203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:41.455678 29202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:41.455837 29197 server_base.cc:1061] running on GCE node
W20261118 03:06:41.455682 29205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:41.456036 29197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:41.456210 29197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:41.457329 29197 hybrid_clock.cc:648] HybridClock initialized: now 1794971201457315 us; error 28 us; skew 500 ppm
I20261118 03:06:41.458340 29197 webserver.cc:577] Webserver started at http://127.24.137.132:41143/ using document root <none> and password file <none>
I20261118 03:06:41.458511 29197 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:41.458565 29197 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:41.458652 29197 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 03:06:41.459450 29197 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "21a20b6c11b04353bd839425064b3993"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.459748 29197 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "21a20b6c11b04353bd839425064b3993"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.460758 29197 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20261118 03:06:41.461423 29211 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.461591 29197 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:41.461653 29197 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "21a20b6c11b04353bd839425064b3993"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.461706 29197 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:41.470798 29197 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:41.470995 29197 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:41.471093 29197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:41.471273 29197 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:41.471570 29197 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 03:06:41.471599 29197 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.471626 29197 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 03:06:41.471649 29197 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.476866 29197 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.132:36111
I20261118 03:06:41.476925 29324 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.132:36111 every 8 connection(s)
I20261118 03:06:41.477203 29197 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261118 03:06:41.478936 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 29197
I20261118 03:06:41.479028 25126 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20261118 03:06:41.481324 29325 heartbeater.cc:345] Connected to a master server at 127.24.137.190:37643
I20261118 03:06:41.481421 29325 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:41.481613 29325 heartbeater.cc:508] Master 127.24.137.190:37643 requested a full tablet report, sending...
I20261118 03:06:41.481945 28744 ts_manager.cc:210] Registered new tserver with Master: 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111)
I20261118 03:06:41.482282 28744 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.132:44495
I20261118 03:06:41.490878 25126 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261118 03:06:41.496495 28744 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54786:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20261118 03:06:41.503732 29128 tablet_service.cc:1500] Processing CreateTablet for tablet 245f1eb5f0cf43feab7ecb68824828e3 (DEFAULT_TABLE table=TestTable [id=a8f3539b819844608445457511bd7d2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:41.504053 29128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 245f1eb5f0cf43feab7ecb68824828e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:41.504159 29259 tablet_service.cc:1500] Processing CreateTablet for tablet 245f1eb5f0cf43feab7ecb68824828e3 (DEFAULT_TABLE table=TestTable [id=a8f3539b819844608445457511bd7d2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:41.504159 28997 tablet_service.cc:1500] Processing CreateTablet for tablet 245f1eb5f0cf43feab7ecb68824828e3 (DEFAULT_TABLE table=TestTable [id=a8f3539b819844608445457511bd7d2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 03:06:41.504415 29259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 245f1eb5f0cf43feab7ecb68824828e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:41.504424 28997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 245f1eb5f0cf43feab7ecb68824828e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 03:06:41.506258 29342 tablet_bootstrap.cc:492] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Bootstrap starting.
I20261118 03:06:41.506873 29342 tablet_bootstrap.cc:654] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:41.506882 29344 tablet_bootstrap.cc:492] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Bootstrap starting.
I20261118 03:06:41.507266 29342 log.cc:826] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:41.507439 29344 tablet_bootstrap.cc:654] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:41.507702 29344 log.cc:826] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:41.507988 29343 tablet_bootstrap.cc:492] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Bootstrap starting.
I20261118 03:06:41.508026 29342 tablet_bootstrap.cc:492] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: No bootstrap required, opened a new log
I20261118 03:06:41.508075 29342 ts_tablet_manager.cc:1449] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261118 03:06:41.508565 29343 tablet_bootstrap.cc:654] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Neither blocks nor log segments found. Creating new log.
I20261118 03:06:41.508828 29343 log.cc:826] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:41.509403 29343 tablet_bootstrap.cc:492] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: No bootstrap required, opened a new log
I20261118 03:06:41.509467 29343 ts_tablet_manager.cc:1449] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20261118 03:06:41.509588 29342 raft_consensus.cc:360] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:41.509720 29342 raft_consensus.cc:386] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:41.509756 29342 raft_consensus.cc:748] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc5e1d4991ee44b1a9bb3b6ca7c0e034, State: Initialized, Role: FOLLOWER
I20261118 03:06:41.509856 29342 consensus_queue.cc:261] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:41.509933 29344 tablet_bootstrap.cc:492] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: No bootstrap required, opened a new log
I20261118 03:06:41.509997 29344 ts_tablet_manager.cc:1449] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20261118 03:06:41.510075 29342 ts_tablet_manager.cc:1480] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 03:06:41.510195 29194 heartbeater.cc:500] Master 127.24.137.190:37643 was elected leader, sending a full tablet report...
I20261118 03:06:41.510968 29343 raft_consensus.cc:360] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:41.511096 29343 raft_consensus.cc:386] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:41.511132 29343 raft_consensus.cc:748] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a20b6c11b04353bd839425064b3993, State: Initialized, Role: FOLLOWER
I20261118 03:06:41.511214 29343 consensus_queue.cc:261] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:41.511439 29343 ts_tablet_manager.cc:1480] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 03:06:41.511466 29325 heartbeater.cc:500] Master 127.24.137.190:37643 was elected leader, sending a full tablet report...
I20261118 03:06:41.511773 29344 raft_consensus.cc:360] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:41.511891 29344 raft_consensus.cc:386] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 03:06:41.511919 29344 raft_consensus.cc:748] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9007147e0bff4e8d877a637c02284b26, State: Initialized, Role: FOLLOWER
I20261118 03:06:41.512006 29344 consensus_queue.cc:261] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:41.512218 29344 ts_tablet_manager.cc:1480] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 03:06:41.512256 29063 heartbeater.cc:500] Master 127.24.137.190:37643 was elected leader, sending a full tablet report...
W20261118 03:06:41.529578 29064 tablet.cc:2441] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261118 03:06:41.569048 29195 tablet.cc:2441] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261118 03:06:41.577644 29326 tablet.cc:2441] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 03:06:41.737893 29349 raft_consensus.cc:499] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 03:06:41.738013 29349 raft_consensus.cc:521] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:41.738287 29349 leader_election.cc:290] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9007147e0bff4e8d877a637c02284b26 (127.24.137.130:43619), dc5e1d4991ee44b1a9bb3b6ca7c0e034 (127.24.137.131:37177)
I20261118 03:06:41.741394 29017 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "21a20b6c11b04353bd839425064b3993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9007147e0bff4e8d877a637c02284b26" is_pre_election: true
I20261118 03:06:41.741588 29017 raft_consensus.cc:2490] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a20b6c11b04353bd839425064b3993 in term 0.
I20261118 03:06:41.741734 29148 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "21a20b6c11b04353bd839425064b3993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" is_pre_election: true
I20261118 03:06:41.741873 29148 raft_consensus.cc:2490] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a20b6c11b04353bd839425064b3993 in term 0.
I20261118 03:06:41.742002 29212 leader_election.cc:304] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a20b6c11b04353bd839425064b3993, 9007147e0bff4e8d877a637c02284b26; no voters:
I20261118 03:06:41.742136 29349 raft_consensus.cc:2826] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 03:06:41.742187 29349 raft_consensus.cc:499] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 03:06:41.742201 29349 raft_consensus.cc:3096] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:41.742811 29349 raft_consensus.cc:521] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:41.742928 29349 leader_election.cc:290] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [CANDIDATE]: Term 1 election: Requested vote from peers 9007147e0bff4e8d877a637c02284b26 (127.24.137.130:43619), dc5e1d4991ee44b1a9bb3b6ca7c0e034 (127.24.137.131:37177)
I20261118 03:06:41.743073 29017 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "21a20b6c11b04353bd839425064b3993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9007147e0bff4e8d877a637c02284b26"
I20261118 03:06:41.743153 29017 raft_consensus.cc:3096] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:41.743134 29148 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "21a20b6c11b04353bd839425064b3993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034"
I20261118 03:06:41.743203 29148 raft_consensus.cc:3096] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 0 FOLLOWER]: Advancing to term 1
I20261118 03:06:41.743764 29017 raft_consensus.cc:2490] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a20b6c11b04353bd839425064b3993 in term 1.
I20261118 03:06:41.743929 29212 leader_election.cc:304] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a20b6c11b04353bd839425064b3993, 9007147e0bff4e8d877a637c02284b26; no voters:
I20261118 03:06:41.744026 29349 raft_consensus.cc:2826] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 03:06:41.744190 29349 raft_consensus.cc:705] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 1 LEADER]: Becoming Leader. State: Replica: 21a20b6c11b04353bd839425064b3993, State: Running, Role: LEADER
I20261118 03:06:41.744180 29148 raft_consensus.cc:2490] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a20b6c11b04353bd839425064b3993 in term 1.
I20261118 03:06:41.744287 29349 consensus_queue.cc:238] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:41.745085 28743 catalog_manager.cc:5840] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a20b6c11b04353bd839425064b3993 (127.24.137.132). New cstate: current_term: 1 leader_uuid: "21a20b6c11b04353bd839425064b3993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } health_report { overall_health: UNKNOWN } } }
I20261118 03:06:41.745378 29354 consensus_queue.cc:1049] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 03:06:41.745488 29349 consensus_queue.cc:1049] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 03:06:41.782492 25126 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20261118 03:06:41.783001 25126 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 9007147e0bff4e8d877a637c02284b26 to finish bootstrapping
I20261118 03:06:41.786470 25126 ts_itest-base.cc:246] Waiting for 1 tablets on tserver dc5e1d4991ee44b1a9bb3b6ca7c0e034 to finish bootstrapping
I20261118 03:06:41.789217 25126 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 21a20b6c11b04353bd839425064b3993 to finish bootstrapping
I20261118 03:06:41.791895 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 29197
I20261118 03:06:41.797183 25126 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskKaSa1L/build/release/bin/kudu
/tmp/dist-test-taskKaSa1L/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.137.132:36111
--local_ip_for_outbound_sockets=127.24.137.132
--tserver_master_addrs=127.24.137.190:37643
--webserver_port=41143
--webserver_interface=127.24.137.132
--builtin_ntp_servers=127.24.137.148:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000
--tablet_bootstrap_inject_latency_ms=3000 with env {}
W20261118 03:06:41.872591 29361 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20261118 03:06:41.872756 29361 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 03:06:41.872769 29361 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 03:06:41.872787 29361 flags.cc:432] Enabled unsafe flag: --tablet_bootstrap_inject_latency_ms=3000
W20261118 03:06:41.872810 29361 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 03:06:41.874400 29361 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20261118 03:06:41.874446 29361 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20261118 03:06:41.874461 29361 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 03:06:41.874488 29361 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20261118 03:06:41.874514 29361 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.137.132
I20261118 03:06:41.876096 29361 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.137.148:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--follower_unavailable_considered_failed_sec=3
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.137.132:36111
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.137.132
--webserver_port=41143
--tserver_master_addrs=127.24.137.190:37643
--tablet_bootstrap_inject_latency_ms=3000
--never_fsync=true
--heap_profile_path=/tmp/kudu.29361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.137.132
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type RELEASE
built by None at 18 Nov 2026 02:43:13 UTC on e7f111948823
build id 14836
I20261118 03:06:41.876276 29361 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 03:06:41.876444 29361 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 03:06:41.878602 29369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:41.878612 29366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 03:06:41.878633 29367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 03:06:41.878780 29361 server_base.cc:1061] running on GCE node
I20261118 03:06:41.878971 29361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 03:06:41.879110 29361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 03:06:41.880225 29361 hybrid_clock.cc:648] HybridClock initialized: now 1794971201880211 us; error 27 us; skew 500 ppm
I20261118 03:06:41.881283 29361 webserver.cc:577] Webserver started at http://127.24.137.132:41143/ using document root <none> and password file <none>
I20261118 03:06:41.881458 29361 fs_manager.cc:362] Metadata directory not provided
I20261118 03:06:41.881510 29361 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 03:06:41.882604 29361 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20261118 03:06:41.883216 29375 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 03:06:41.883379 29361 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20261118 03:06:41.883436 29361 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "21a20b6c11b04353bd839425064b3993"
format_stamp: "Formatted at 2026-11-18 03:06:41 on dist-test-slave-k60w"
I20261118 03:06:41.883692 29361 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 03:06:41.903331 29361 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 03:06:41.903558 29361 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 03:06:41.903661 29361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 03:06:41.903838 29361 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 03:06:41.904277 29382 ts_tablet_manager.cc:588] Loading tablet metadata (0/1 complete)
I20261118 03:06:41.905107 29361 ts_tablet_manager.cc:631] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20261118 03:06:41.905144 29361 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20261118 03:06:41.905177 29361 ts_tablet_manager.cc:646] Registering tablets (0/1 complete)
I20261118 03:06:41.905743 29361 ts_tablet_manager.cc:662] Registered 1 tablets
W20261118 03:06:41.905766 29382 ts_tablet_manager.cc:1428] Injecting 3000ms delay in tablet bootstrapping
I20261118 03:06:41.905774 29361 ts_tablet_manager.cc:641] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20261118 03:06:41.911343 29361 rpc_server.cc:307] RPC server started. Bound to: 127.24.137.132:36111
I20261118 03:06:41.911370 29489 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.137.132:36111 every 8 connection(s)
I20261118 03:06:41.911725 29361 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20261118 03:06:41.916117 29490 heartbeater.cc:345] Connected to a master server at 127.24.137.190:37643
I20261118 03:06:41.916216 29490 heartbeater.cc:462] Registering TS with master...
I20261118 03:06:41.916414 29490 heartbeater.cc:508] Master 127.24.137.190:37643 requested a full tablet report, sending...
I20261118 03:06:41.916906 28744 ts_manager.cc:210] Re-registered known tserver with Master: 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111)
I20261118 03:06:41.917255 28744 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.24.137.132:37199
I20261118 03:06:41.921824 25126 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskKaSa1L/build/release/bin/kudu as pid 29361
I20261118 03:06:42.101277 28932 heartbeater.cc:500] Master 127.24.137.190:37643 was elected leader, sending a full tablet report...
I20261118 03:06:42.917989 29490 heartbeater.cc:500] Master 127.24.137.190:37643 was elected leader, sending a full tablet report...
I20261118 03:06:43.247982 29496 raft_consensus.cc:499] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 21a20b6c11b04353bd839425064b3993)
I20261118 03:06:43.247985 29495 raft_consensus.cc:499] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 21a20b6c11b04353bd839425064b3993)
I20261118 03:06:43.248075 29496 raft_consensus.cc:521] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:43.248075 29495 raft_consensus.cc:521] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:43.248334 29495 leader_election.cc:290] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111), dc5e1d4991ee44b1a9bb3b6ca7c0e034 (127.24.137.131:37177)
I20261118 03:06:43.248586 29496 leader_election.cc:290] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9007147e0bff4e8d877a637c02284b26 (127.24.137.130:43619), 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111)
I20261118 03:06:43.251991 29017 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9007147e0bff4e8d877a637c02284b26" is_pre_election: true
I20261118 03:06:43.252090 29017 raft_consensus.cc:2490] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc5e1d4991ee44b1a9bb3b6ca7c0e034 in term 1.
I20261118 03:06:43.252269 29148 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "9007147e0bff4e8d877a637c02284b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" is_pre_election: true
I20261118 03:06:43.252286 29081 leader_election.cc:304] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9007147e0bff4e8d877a637c02284b26, dc5e1d4991ee44b1a9bb3b6ca7c0e034; no voters:
I20261118 03:06:43.252365 29148 raft_consensus.cc:2490] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9007147e0bff4e8d877a637c02284b26 in term 1.
I20261118 03:06:43.252418 29496 raft_consensus.cc:2826] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261118 03:06:43.252451 29496 raft_consensus.cc:499] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 21a20b6c11b04353bd839425064b3993)
I20261118 03:06:43.252470 29496 raft_consensus.cc:3096] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 1 FOLLOWER]: Advancing to term 2
I20261118 03:06:43.252523 28950 leader_election.cc:304] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9007147e0bff4e8d877a637c02284b26, dc5e1d4991ee44b1a9bb3b6ca7c0e034; no voters:
I20261118 03:06:43.252622 29495 raft_consensus.cc:2826] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261118 03:06:43.252655 29495 raft_consensus.cc:499] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 21a20b6c11b04353bd839425064b3993)
I20261118 03:06:43.252676 29495 raft_consensus.cc:3096] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 1 FOLLOWER]: Advancing to term 2
I20261118 03:06:43.252177 29444 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "21a20b6c11b04353bd839425064b3993" is_pre_election: true
I20261118 03:06:43.252169 29443 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "9007147e0bff4e8d877a637c02284b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "21a20b6c11b04353bd839425064b3993" is_pre_election: true
W20261118 03:06:43.253217 28951 leader_election.cc:343] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111): Illegal state: must be running to vote when last-logged opid is not known
I20261118 03:06:43.253270 29496 raft_consensus.cc:521] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:43.253301 29495 raft_consensus.cc:521] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:43.253405 29496 leader_election.cc:290] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [CANDIDATE]: Term 2 election: Requested vote from peers 9007147e0bff4e8d877a637c02284b26 (127.24.137.130:43619), 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111)
I20261118 03:06:43.253445 29495 leader_election.cc:290] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 2 election: Requested vote from peers 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111), dc5e1d4991ee44b1a9bb3b6ca7c0e034 (127.24.137.131:37177)
I20261118 03:06:43.253599 29017 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9007147e0bff4e8d877a637c02284b26"
I20261118 03:06:43.253592 29148 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "9007147e0bff4e8d877a637c02284b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034"
I20261118 03:06:43.253664 29017 raft_consensus.cc:2415] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dc5e1d4991ee44b1a9bb3b6ca7c0e034 in current term 2: Already voted for candidate 9007147e0bff4e8d877a637c02284b26 in this term.
I20261118 03:06:43.253664 29148 raft_consensus.cc:2415] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9007147e0bff4e8d877a637c02284b26 in current term 2: Already voted for candidate dc5e1d4991ee44b1a9bb3b6ca7c0e034 in this term.
I20261118 03:06:43.253623 29443 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "9007147e0bff4e8d877a637c02284b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "21a20b6c11b04353bd839425064b3993"
I20261118 03:06:43.253623 29444 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "21a20b6c11b04353bd839425064b3993"
W20261118 03:06:43.253791 28951 leader_election.cc:343] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111): Illegal state: must be running to vote when last-logged opid is not known
I20261118 03:06:43.253829 28951 leader_election.cc:304] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9007147e0bff4e8d877a637c02284b26; no voters: 21a20b6c11b04353bd839425064b3993, dc5e1d4991ee44b1a9bb3b6ca7c0e034
W20261118 03:06:43.253809 29082 leader_election.cc:343] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111): Illegal state: must be running to vote when last-logged opid is not known
I20261118 03:06:43.253866 29081 leader_election.cc:304] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc5e1d4991ee44b1a9bb3b6ca7c0e034; no voters: 21a20b6c11b04353bd839425064b3993, 9007147e0bff4e8d877a637c02284b26
I20261118 03:06:43.253901 29495 raft_consensus.cc:2771] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20261118 03:06:43.253939 29496 raft_consensus.cc:2771] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20261118 03:06:44.853314 29505 raft_consensus.cc:499] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 03:06:44.853413 29505 raft_consensus.cc:521] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:44.853619 29505 leader_election.cc:290] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111), dc5e1d4991ee44b1a9bb3b6ca7c0e034 (127.24.137.131:37177)
I20261118 03:06:44.853852 29148 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "9007147e0bff4e8d877a637c02284b26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" is_pre_election: true
I20261118 03:06:44.853850 29443 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "9007147e0bff4e8d877a637c02284b26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "21a20b6c11b04353bd839425064b3993" is_pre_election: true
I20261118 03:06:44.853948 29148 raft_consensus.cc:2490] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9007147e0bff4e8d877a637c02284b26 in term 2.
W20261118 03:06:44.854058 28951 leader_election.cc:343] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111): Illegal state: must be running to vote when last-logged opid is not known
I20261118 03:06:44.854116 28950 leader_election.cc:304] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9007147e0bff4e8d877a637c02284b26, dc5e1d4991ee44b1a9bb3b6ca7c0e034; no voters: 21a20b6c11b04353bd839425064b3993
I20261118 03:06:44.854218 29505 raft_consensus.cc:2826] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20261118 03:06:44.854261 29505 raft_consensus.cc:499] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 03:06:44.854286 29505 raft_consensus.cc:3096] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 2 FOLLOWER]: Advancing to term 3
I20261118 03:06:44.857060 29505 raft_consensus.cc:521] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:44.857211 29505 leader_election.cc:290] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 3 election: Requested vote from peers 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111), dc5e1d4991ee44b1a9bb3b6ca7c0e034 (127.24.137.131:37177)
I20261118 03:06:44.857409 29148 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "9007147e0bff4e8d877a637c02284b26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034"
I20261118 03:06:44.857411 29443 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "245f1eb5f0cf43feab7ecb68824828e3" candidate_uuid: "9007147e0bff4e8d877a637c02284b26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "21a20b6c11b04353bd839425064b3993"
I20261118 03:06:44.857488 29148 raft_consensus.cc:3096] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 2 FOLLOWER]: Advancing to term 3
W20261118 03:06:44.857646 28951 leader_election.cc:343] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111): Illegal state: must be running to vote when last-logged opid is not known
I20261118 03:06:44.858100 29148 raft_consensus.cc:2490] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9007147e0bff4e8d877a637c02284b26 in term 3.
I20261118 03:06:44.858260 28950 leader_election.cc:304] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9007147e0bff4e8d877a637c02284b26, dc5e1d4991ee44b1a9bb3b6ca7c0e034; no voters: 21a20b6c11b04353bd839425064b3993
I20261118 03:06:44.858363 29505 raft_consensus.cc:2826] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 3 FOLLOWER]: Leader election won for term 3
I20261118 03:06:44.858493 29505 raft_consensus.cc:705] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [term 3 LEADER]: Becoming Leader. State: Replica: 9007147e0bff4e8d877a637c02284b26, State: Running, Role: LEADER
I20261118 03:06:44.858577 29505 consensus_queue.cc:238] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:44.859210 29148 raft_consensus.cc:1285] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034 [term 3 FOLLOWER]: Refusing update from remote peer 9007147e0bff4e8d877a637c02284b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20261118 03:06:44.859206 28743 catalog_manager.cc:5840] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 reported cstate change: term changed from 1 to 3, leader changed from 21a20b6c11b04353bd839425064b3993 (127.24.137.132) to 9007147e0bff4e8d877a637c02284b26 (127.24.137.130). New cstate: current_term: 3 leader_uuid: "9007147e0bff4e8d877a637c02284b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } health_report { overall_health: UNKNOWN } } }
I20261118 03:06:44.859400 29505 consensus_queue.cc:1049] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20261118 03:06:44.859591 28951 consensus_peers.cc:597] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26 -> Peer 21a20b6c11b04353bd839425064b3993 (127.24.137.132:36111): Couldn't send request to peer 21a20b6c11b04353bd839425064b3993. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20261118 03:06:44.906006 29382 tablet_bootstrap.cc:492] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Bootstrap starting.
I20261118 03:06:44.908696 29382 log.cc:826] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Log is configured to *not* fsync() on all Append() calls
I20261118 03:06:44.909518 29382 tablet_bootstrap.cc:492] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 03:06:44.909812 29382 tablet_bootstrap.cc:492] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Bootstrap complete.
I20261118 03:06:44.909981 29382 ts_tablet_manager.cc:1449] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20261118 03:06:44.910629 29382 raft_consensus.cc:360] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:44.910801 29382 raft_consensus.cc:748] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a20b6c11b04353bd839425064b3993, State: Initialized, Role: FOLLOWER
I20261118 03:06:44.910884 29382 consensus_queue.cc:261] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9007147e0bff4e8d877a637c02284b26" member_type: VOTER last_known_addr { host: "127.24.137.130" port: 43619 } } peers { permanent_uuid: "21a20b6c11b04353bd839425064b3993" member_type: VOTER last_known_addr { host: "127.24.137.132" port: 36111 } } peers { permanent_uuid: "dc5e1d4991ee44b1a9bb3b6ca7c0e034" member_type: VOTER last_known_addr { host: "127.24.137.131" port: 37177 } }
I20261118 03:06:44.911085 29382 ts_tablet_manager.cc:1480] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20261118 03:06:45.334844 29443 raft_consensus.cc:3096] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993 [term 1 FOLLOWER]: Advancing to term 3
I20261118 03:06:45.556665 25126 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20261118 03:06:45.556915 25126 raft_config_change-itest.cc:610] Waiting until we've written at least 4MB...
I20261118 03:06:45.603127 25126 raft_config_change-itest.cc:616] Waiting for log GC on 9007147e0bff4e8d877a637c02284b26
I20261118 03:06:45.618044 29491 maintenance_manager.cc:419] P 21a20b6c11b04353bd839425064b3993: Scheduling FlushMRSOp(245f1eb5f0cf43feab7ecb68824828e3): 4076156 bytes log retention, and flush 6119689 bytes memory
I20261118 03:06:45.623031 29380 maintenance_manager.cc:643] P 21a20b6c11b04353bd839425064b3993: FlushMRSOp(245f1eb5f0cf43feab7ecb68824828e3) complete. Timing: real 0.005s user 0.004s sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":140,"dirs.queue_time_us":174,"dirs.run_cpu_time_us":171,"dirs.run_wall_time_us":668,"drs_written":1,"lbm_read_time_us":20,"lbm_reads_lt_1ms":4,"lbm_write_time_us":199,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":38,"thread_start_us":192,"threads_started":2}
I20261118 03:06:45.623265 29491 maintenance_manager.cc:419] P 21a20b6c11b04353bd839425064b3993: Scheduling LogGCOp(245f1eb5f0cf43feab7ecb68824828e3): free 4076156 bytes of WAL
I20261118 03:06:45.623370 29380 log_reader.cc:385] T 245f1eb5f0cf43feab7ecb68824828e3: removed 5 log segments from log reader
I20261118 03:06:45.623415 29380 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000001 (ops 1-8)
I20261118 03:06:45.623437 29380 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000002 (ops 9-15)
I20261118 03:06:45.623459 29380 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000003 (ops 16-21)
I20261118 03:06:45.623481 29380 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000004 (ops 22-26)
I20261118 03:06:45.623502 29380 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P 21a20b6c11b04353bd839425064b3993: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-3/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000005 (ops 27-33)
I20261118 03:06:45.624617 29380 maintenance_manager.cc:643] P 21a20b6c11b04353bd839425064b3993: LogGCOp(245f1eb5f0cf43feab7ecb68824828e3) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261118 03:06:45.624778 29491 maintenance_manager.cc:419] P 21a20b6c11b04353bd839425064b3993: Scheduling UndoDeltaBlockGCOp(245f1eb5f0cf43feab7ecb68824828e3): 558 bytes on disk
I20261118 03:06:45.624970 29380 maintenance_manager.cc:643] P 21a20b6c11b04353bd839425064b3993: UndoDeltaBlockGCOp(245f1eb5f0cf43feab7ecb68824828e3) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":22,"lbm_reads_lt_1ms":4}
I20261118 03:06:45.638622 29064 maintenance_manager.cc:419] P 9007147e0bff4e8d877a637c02284b26: Scheduling FlushMRSOp(245f1eb5f0cf43feab7ecb68824828e3): 4602312 bytes log retention, and flush 6119689 bytes memory
I20261118 03:06:45.644764 28954 maintenance_manager.cc:643] P 9007147e0bff4e8d877a637c02284b26: FlushMRSOp(245f1eb5f0cf43feab7ecb68824828e3) complete. Timing: real 0.006s user 0.005s sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":154,"dirs.queue_time_us":173,"dirs.run_cpu_time_us":175,"dirs.run_wall_time_us":746,"drs_written":1,"lbm_read_time_us":20,"lbm_reads_lt_1ms":4,"lbm_write_time_us":280,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":38,"thread_start_us":221,"threads_started":2}
I20261118 03:06:45.644965 29064 maintenance_manager.cc:419] P 9007147e0bff4e8d877a637c02284b26: Scheduling LogGCOp(245f1eb5f0cf43feab7ecb68824828e3): free 4602312 bytes of WAL
I20261118 03:06:45.645089 28954 log_reader.cc:385] T 245f1eb5f0cf43feab7ecb68824828e3: removed 5 log segments from log reader
I20261118 03:06:45.645126 28954 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000001 (ops 1-9)
I20261118 03:06:45.645147 28954 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000002 (ops 10-16)
I20261118 03:06:45.645186 28954 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000003 (ops 17-23)
I20261118 03:06:45.645227 28954 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000004 (ops 24-30)
I20261118 03:06:45.645255 28954 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P 9007147e0bff4e8d877a637c02284b26: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-1/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000005 (ops 31-37)
I20261118 03:06:45.646634 28954 maintenance_manager.cc:643] P 9007147e0bff4e8d877a637c02284b26: LogGCOp(245f1eb5f0cf43feab7ecb68824828e3) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {}
I20261118 03:06:45.646798 29064 maintenance_manager.cc:419] P 9007147e0bff4e8d877a637c02284b26: Scheduling UndoDeltaBlockGCOp(245f1eb5f0cf43feab7ecb68824828e3): 558 bytes on disk
I20261118 03:06:45.646978 28954 maintenance_manager.cc:643] P 9007147e0bff4e8d877a637c02284b26: UndoDeltaBlockGCOp(245f1eb5f0cf43feab7ecb68824828e3) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":23,"lbm_reads_lt_1ms":4}
I20261118 03:06:45.654201 25126 raft_config_change-itest.cc:620] Log GC complete on 9007147e0bff4e8d877a637c02284b26
I20261118 03:06:45.676126 29195 maintenance_manager.cc:419] P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Scheduling FlushMRSOp(245f1eb5f0cf43feab7ecb68824828e3): 4076188 bytes log retention, and flush 6119689 bytes memory
I20261118 03:06:45.682502 29085 maintenance_manager.cc:643] P dc5e1d4991ee44b1a9bb3b6ca7c0e034: FlushMRSOp(245f1eb5f0cf43feab7ecb68824828e3) complete. Timing: real 0.006s user 0.001s sys 0.004s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":164,"drs_written":1,"lbm_read_time_us":20,"lbm_reads_lt_1ms":4,"lbm_write_time_us":292,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":38,"thread_start_us":228,"threads_started":2}
I20261118 03:06:45.682766 29195 maintenance_manager.cc:419] P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Scheduling LogGCOp(245f1eb5f0cf43feab7ecb68824828e3): free 4076188 bytes of WAL
I20261118 03:06:45.682967 29085 log_reader.cc:385] T 245f1eb5f0cf43feab7ecb68824828e3: removed 5 log segments from log reader
I20261118 03:06:45.683004 29085 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000001 (ops 1-9)
I20261118 03:06:45.683025 29085 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000002 (ops 10-15)
I20261118 03:06:45.683043 29085 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000003 (ops 16-22)
I20261118 03:06:45.683058 29085 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000004 (ops 23-27)
I20261118 03:06:45.683074 29085 log.cc:1079] T 245f1eb5f0cf43feab7ecb68824828e3 P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Deleting log segment in path: /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0/raft_consensus-itest-cluster/ts-2/wal/wals/245f1eb5f0cf43feab7ecb68824828e3/wal-000000005 (ops 28-33)
I20261118 03:06:45.684226 29085 maintenance_manager.cc:643] P dc5e1d4991ee44b1a9bb3b6ca7c0e034: LogGCOp(245f1eb5f0cf43feab7ecb68824828e3) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20261118 03:06:45.684461 29195 maintenance_manager.cc:419] P dc5e1d4991ee44b1a9bb3b6ca7c0e034: Scheduling UndoDeltaBlockGCOp(245f1eb5f0cf43feab7ecb68824828e3): 558 bytes on disk
I20261118 03:06:45.684669 29085 maintenance_manager.cc:643] P dc5e1d4991ee44b1a9bb3b6ca7c0e034: UndoDeltaBlockGCOp(245f1eb5f0cf43feab7ecb68824828e3) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":22,"lbm_reads_lt_1ms":4}
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/integration-tests/raft_config_change-itest.cc:500: Failure
Value of: IsRaftConfigMember(added_replica_uuid, cstate.committed_config())
Actual: false
Expected: true
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20261118 03:07:15.731773 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 28804
I20261118 03:07:15.736196 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 28935
I20261118 03:07:15.741096 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 29066
I20261118 03:07:15.746040 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 29361
I20261118 03:07:15.750654 25126 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskKaSa1L/build/release/bin/kudu with pid 28713
2026-11-18T03:07:15Z chronyd exiting
I20261118 03:07:15.766670 25126 test_util.cc:182] -----------------------------------------------
I20261118 03:07:15.766742 25126 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskKaSa1L/test-tmp/raft_config_change-itest.0.SlowTabletBootstrapTest.TestFallBehindSlowBootstrap.1794971175865063-25126-0
[ FAILED ] SlowTabletBootstrapTest.TestFallBehindSlowBootstrap (34936 ms)
[----------] 3 tests from SlowTabletBootstrapTest (46096 ms total)
[----------] Global test environment tear-down
[==========] 6 tests from 2 test suites ran. (59897 ms total)
[ PASSED ] 5 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SlowTabletBootstrapTest.TestFallBehindSlowBootstrap
1 FAILED TEST