Note: This is test shard 6 of 8.
[==========] Running 9 tests from 5 test suites.
[----------] Global test environment set-up.
[----------] 5 tests from AdminCliTest
[ RUN ] AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250815 02:08:35.468000 6382 test_util.cc:276] Using random seed: -569388336
I20250815 02:08:35.472991 6382 ts_itest-base.cc:115] Starting cluster with:
I20250815 02:08:35.473037 6382 ts_itest-base.cc:116] --------------
I20250815 02:08:35.473057 6382 ts_itest-base.cc:117] 4 tablet servers
I20250815 02:08:35.473075 6382 ts_itest-base.cc:118] 3 replicas per TS
I20250815 02:08:35.473090 6382 ts_itest-base.cc:119] --------------
2025-08-15T02:08:35Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-15T02:08:35Z Disabled control of system clock
I20250815 02:08:35.480517 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:41647
--webserver_interface=127.6.59.190
--webserver_port=0
--builtin_ntp_servers=127.6.59.148:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:41647 with env {}
W20250815 02:08:35.557835 6394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:35.558027 6394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:35.558046 6394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:35.559429 6394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:35.559469 6394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:35.559484 6394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:35.559494 6394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:35.560959 6394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:41647
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:41647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.6394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:35.561132 6394 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:35.561318 6394 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:35.563947 6400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:35.563962 6402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:35.563962 6399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:35.564235 6394 server_base.cc:1047] running on GCE node
I20250815 02:08:35.564435 6394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:35.564682 6394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:35.565815 6394 hybrid_clock.cc:648] HybridClock initialized: now 1755223715565805 us; error 24 us; skew 500 ppm
I20250815 02:08:35.567013 6394 webserver.cc:480] Webserver started at http://127.6.59.190:45173/ using document root <none> and password file <none>
I20250815 02:08:35.567202 6394 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:35.567248 6394 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:35.567361 6394 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:35.568205 6394 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "2478edf97ac04278bc86d7ed81148566"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.568486 6394 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "2478edf97ac04278bc86d7ed81148566"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.569739 6394 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20250815 02:08:35.570458 6408 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:35.570624 6394 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:35.570705 6394 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
uuid: "2478edf97ac04278bc86d7ed81148566"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.570762 6394 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:35.603257 6394 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:35.603530 6394 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:35.603637 6394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:35.607120 6394 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:41647
I20250815 02:08:35.607196 6460 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:41647 every 8 connection(s)
I20250815 02:08:35.607465 6394 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250815 02:08:35.608088 6461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:35.610306 6461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566: Bootstrap starting.
I20250815 02:08:35.610870 6461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:35.611115 6461 log.cc:826] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:35.611655 6461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566: No bootstrap required, opened a new log
I20250815 02:08:35.612735 6461 raft_consensus.cc:357] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } }
I20250815 02:08:35.612851 6461 raft_consensus.cc:383] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:35.612870 6461 raft_consensus.cc:738] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2478edf97ac04278bc86d7ed81148566, State: Initialized, Role: FOLLOWER
I20250815 02:08:35.613008 6461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } }
I20250815 02:08:35.613065 6461 raft_consensus.cc:397] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:35.613088 6461 raft_consensus.cc:491] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:35.613134 6461 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:35.613603 6461 raft_consensus.cc:513] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } }
I20250815 02:08:35.613696 6461 leader_election.cc:304] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2478edf97ac04278bc86d7ed81148566; no voters:
I20250815 02:08:35.613859 6461 leader_election.cc:290] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:35.613919 6464 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:35.614084 6464 raft_consensus.cc:695] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 1 LEADER]: Becoming Leader. State: Replica: 2478edf97ac04278bc86d7ed81148566, State: Running, Role: LEADER
I20250815 02:08:35.614171 6464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } }
I20250815 02:08:35.614223 6461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:35.614511 6465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2478edf97ac04278bc86d7ed81148566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } } }
I20250815 02:08:35.614715 6465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:35.614660 6466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2478edf97ac04278bc86d7ed81148566. Latest consensus state: current_term: 1 leader_uuid: "2478edf97ac04278bc86d7ed81148566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } } }
I20250815 02:08:35.614774 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 6394
I20250815 02:08:35.614877 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250815 02:08:35.614903 6466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:35.615204 6469 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:35.615649 6469 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:35.618544 6469 catalog_manager.cc:1349] Generated new cluster ID: 114ca840588b471989aa05ad6b1517d3
I20250815 02:08:35.618593 6469 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:35.636696 6469 catalog_manager.cc:1372] Generated new certificate authority record
I20250815 02:08:35.637219 6469 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:35.642518 6469 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566: Generated new TSK 0
I20250815 02:08:35.642678 6469 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250815 02:08:35.646833 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.129:0
--local_ip_for_outbound_sockets=127.6.59.129
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:41647
--builtin_ntp_servers=127.6.59.148:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:35.721167 6485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:35.721325 6485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:35.721346 6485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:35.722739 6485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:35.722787 6485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.129
I20250815 02:08:35.724231 6485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:41647
--never_fsync=true
--heap_profile_path=/tmp/kudu.6485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.129
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:35.724433 6485 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:35.724664 6485 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:35.727124 6491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:35.727283 6490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:35.727392 6493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:35.727288 6485 server_base.cc:1047] running on GCE node
I20250815 02:08:35.727690 6485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:35.727957 6485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:35.729136 6485 hybrid_clock.cc:648] HybridClock initialized: now 1755223715729114 us; error 37 us; skew 500 ppm
I20250815 02:08:35.730475 6485 webserver.cc:480] Webserver started at http://127.6.59.129:39967/ using document root <none> and password file <none>
I20250815 02:08:35.730772 6485 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:35.730821 6485 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:35.730937 6485 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:35.731774 6485 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "7ac0377c4adb40159b3c9ba9c1e2437b"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.732057 6485 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "7ac0377c4adb40159b3c9ba9c1e2437b"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.733158 6485 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:35.733800 6499 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:35.734010 6485 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:35.734098 6485 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "7ac0377c4adb40159b3c9ba9c1e2437b"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.734225 6485 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:35.745469 6485 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:35.745719 6485 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:35.745826 6485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:35.746080 6485 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:35.746404 6485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:35.746443 6485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:35.746472 6485 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:35.746488 6485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:35.753264 6485 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.129:42099
I20250815 02:08:35.753304 6612 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.129:42099 every 8 connection(s)
I20250815 02:08:35.753715 6485 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250815 02:08:35.758348 6613 heartbeater.cc:344] Connected to a master server at 127.6.59.190:41647
I20250815 02:08:35.758448 6613 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:35.758633 6613 heartbeater.cc:507] Master 127.6.59.190:41647 requested a full tablet report, sending...
I20250815 02:08:35.759105 6425 ts_manager.cc:194] Registered new tserver with Master: 7ac0377c4adb40159b3c9ba9c1e2437b (127.6.59.129:42099)
I20250815 02:08:35.759969 6425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.129:43103
I20250815 02:08:35.760943 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 6485
I20250815 02:08:35.761015 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250815 02:08:35.762003 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.130:0
--local_ip_for_outbound_sockets=127.6.59.130
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:41647
--builtin_ntp_servers=127.6.59.148:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:35.839649 6616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:35.839812 6616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:35.839833 6616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:35.841167 6616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:35.841218 6616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.130
I20250815 02:08:35.842658 6616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:41647
--never_fsync=true
--heap_profile_path=/tmp/kudu.6616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.130
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:35.842870 6616 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:35.843078 6616 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:35.845607 6622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:35.845613 6624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:35.845623 6621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:35.845685 6616 server_base.cc:1047] running on GCE node
I20250815 02:08:35.846024 6616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:35.846226 6616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:35.847363 6616 hybrid_clock.cc:648] HybridClock initialized: now 1755223715847350 us; error 28 us; skew 500 ppm
I20250815 02:08:35.848479 6616 webserver.cc:480] Webserver started at http://127.6.59.130:44309/ using document root <none> and password file <none>
I20250815 02:08:35.848678 6616 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:35.848722 6616 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:35.848832 6616 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:35.849643 6616 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "bcfeb2efeec043948c8707b177881f0a"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.849928 6616 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "bcfeb2efeec043948c8707b177881f0a"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.851058 6616 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:35.851709 6630 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:35.851864 6616 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:35.851944 6616 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "bcfeb2efeec043948c8707b177881f0a"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.852000 6616 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:35.870421 6616 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:35.870664 6616 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:35.870775 6616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:35.870987 6616 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:35.871301 6616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:35.871335 6616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:35.871371 6616 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:35.871399 6616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:35.877182 6616 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.130:36313
I20250815 02:08:35.877238 6743 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.130:36313 every 8 connection(s)
I20250815 02:08:35.877518 6616 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250815 02:08:35.881479 6744 heartbeater.cc:344] Connected to a master server at 127.6.59.190:41647
I20250815 02:08:35.881567 6744 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:35.881733 6744 heartbeater.cc:507] Master 127.6.59.190:41647 requested a full tablet report, sending...
I20250815 02:08:35.882036 6425 ts_manager.cc:194] Registered new tserver with Master: bcfeb2efeec043948c8707b177881f0a (127.6.59.130:36313)
I20250815 02:08:35.882385 6425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.130:48511
I20250815 02:08:35.886149 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 6616
I20250815 02:08:35.886219 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250815 02:08:35.887116 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.131:0
--local_ip_for_outbound_sockets=127.6.59.131
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:41647
--builtin_ntp_servers=127.6.59.148:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:35.964951 6747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:35.965096 6747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:35.965113 6747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:35.966287 6747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:35.966328 6747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.131
I20250815 02:08:35.967693 6747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:41647
--never_fsync=true
--heap_profile_path=/tmp/kudu.6747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.131
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:35.967864 6747 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:35.968040 6747 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:35.970381 6752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:35.970412 6755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:35.970423 6747 server_base.cc:1047] running on GCE node
W20250815 02:08:35.970397 6753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:35.970773 6747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:35.970970 6747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:35.972093 6747 hybrid_clock.cc:648] HybridClock initialized: now 1755223715972075 us; error 35 us; skew 500 ppm
I20250815 02:08:35.973160 6747 webserver.cc:480] Webserver started at http://127.6.59.131:40489/ using document root <none> and password file <none>
I20250815 02:08:35.973348 6747 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:35.973387 6747 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:35.973495 6747 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:35.974361 6747 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.974658 6747 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.975771 6747 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:35.976415 6761 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:35.976563 6747 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:35.976625 6747 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:35.976691 6747 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:35.998416 6747 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:35.998662 6747 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:35.998778 6747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:35.999034 6747 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:35.999375 6747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:35.999406 6747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:35.999432 6747 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:35.999461 6747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:36.005419 6747 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.131:34477
I20250815 02:08:36.005476 6874 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.131:34477 every 8 connection(s)
I20250815 02:08:36.005766 6747 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250815 02:08:36.009917 6875 heartbeater.cc:344] Connected to a master server at 127.6.59.190:41647
I20250815 02:08:36.010036 6875 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:36.010190 6875 heartbeater.cc:507] Master 127.6.59.190:41647 requested a full tablet report, sending...
I20250815 02:08:36.010499 6425 ts_manager.cc:194] Registered new tserver with Master: bba4fd6eb6864cf182fd2fc34cbcd5a8 (127.6.59.131:34477)
I20250815 02:08:36.010726 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 6747
I20250815 02:08:36.010804 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250815 02:08:36.010974 6425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.131:49273
I20250815 02:08:36.011686 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.132:0
--local_ip_for_outbound_sockets=127.6.59.132
--webserver_interface=127.6.59.132
--webserver_port=0
--tserver_master_addrs=127.6.59.190:41647
--builtin_ntp_servers=127.6.59.148:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:36.089334 6878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:36.089485 6878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:36.089504 6878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:36.090910 6878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:36.090970 6878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.132
I20250815 02:08:36.092411 6878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.6.59.132
--webserver_port=0
--tserver_master_addrs=127.6.59.190:41647
--never_fsync=true
--heap_profile_path=/tmp/kudu.6878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.132
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:36.092634 6878 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:36.092857 6878 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:36.095227 6884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:36.095235 6883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:36.095239 6886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:36.095337 6878 server_base.cc:1047] running on GCE node
I20250815 02:08:36.095566 6878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:36.095752 6878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:36.096885 6878 hybrid_clock.cc:648] HybridClock initialized: now 1755223716096871 us; error 29 us; skew 500 ppm
I20250815 02:08:36.097921 6878 webserver.cc:480] Webserver started at http://127.6.59.132:37583/ using document root <none> and password file <none>
I20250815 02:08:36.098142 6878 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:36.098188 6878 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:36.098300 6878 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:36.099133 6878 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "d53365830b6a4d2694d4abd1f91cc22c"
format_stamp: "Formatted at 2025-08-15 02:08:36 on dist-test-slave-wq2q"
I20250815 02:08:36.099416 6878 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "d53365830b6a4d2694d4abd1f91cc22c"
format_stamp: "Formatted at 2025-08-15 02:08:36 on dist-test-slave-wq2q"
I20250815 02:08:36.100492 6878 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20250815 02:08:36.101100 6892 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:36.101251 6878 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:36.101318 6878 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "d53365830b6a4d2694d4abd1f91cc22c"
format_stamp: "Formatted at 2025-08-15 02:08:36 on dist-test-slave-wq2q"
I20250815 02:08:36.101373 6878 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:36.113065 6878 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:36.113312 6878 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:36.113435 6878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:36.113656 6878 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:36.114017 6878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:36.114058 6878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:36.114089 6878 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:36.114156 6878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:36.120517 6878 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.132:36145
I20250815 02:08:36.121023 6878 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20250815 02:08:36.121120 7006 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.132:36145 every 8 connection(s)
I20250815 02:08:36.125699 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 6878
I20250815 02:08:36.125833 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20250815 02:08:36.126320 7007 heartbeater.cc:344] Connected to a master server at 127.6.59.190:41647
I20250815 02:08:36.126418 7007 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:36.126605 7007 heartbeater.cc:507] Master 127.6.59.190:41647 requested a full tablet report, sending...
I20250815 02:08:36.126948 6425 ts_manager.cc:194] Registered new tserver with Master: d53365830b6a4d2694d4abd1f91cc22c (127.6.59.132:36145)
I20250815 02:08:36.127342 6425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.132:53397
I20250815 02:08:36.127417 6382 external_mini_cluster.cc:949] 4 TS(s) registered with all masters
I20250815 02:08:36.133191 6425 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36236:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20250815 02:08:36.140137 6678 tablet_service.cc:1468] Processing CreateTablet for tablet f188c5ac9b434346869e614cd13ce64d (DEFAULT_TABLE table=TestTable [id=e00ac1febb664a9faebe1eda59f05352]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:36.140431 6678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f188c5ac9b434346869e614cd13ce64d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:36.140605 6547 tablet_service.cc:1468] Processing CreateTablet for tablet f188c5ac9b434346869e614cd13ce64d (DEFAULT_TABLE table=TestTable [id=e00ac1febb664a9faebe1eda59f05352]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:36.140847 6547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f188c5ac9b434346869e614cd13ce64d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:36.142982 7021 tablet_bootstrap.cc:492] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b: Bootstrap starting.
I20250815 02:08:36.143265 7022 tablet_bootstrap.cc:492] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a: Bootstrap starting.
I20250815 02:08:36.143647 7021 tablet_bootstrap.cc:654] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:36.143862 7022 tablet_bootstrap.cc:654] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:36.143932 7021 log.cc:826] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:36.144150 7022 log.cc:826] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:36.144147 6809 tablet_service.cc:1468] Processing CreateTablet for tablet f188c5ac9b434346869e614cd13ce64d (DEFAULT_TABLE table=TestTable [id=e00ac1febb664a9faebe1eda59f05352]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:36.144433 6809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f188c5ac9b434346869e614cd13ce64d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:36.144654 7021 tablet_bootstrap.cc:492] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b: No bootstrap required, opened a new log
I20250815 02:08:36.144703 7021 ts_tablet_manager.cc:1397] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:36.144809 7022 tablet_bootstrap.cc:492] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a: No bootstrap required, opened a new log
I20250815 02:08:36.144858 7022 ts_tablet_manager.cc:1397] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:36.145619 7021 raft_consensus.cc:357] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }
I20250815 02:08:36.145721 7021 raft_consensus.cc:383] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:36.145740 7021 raft_consensus.cc:738] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ac0377c4adb40159b3c9ba9c1e2437b, State: Initialized, Role: FOLLOWER
I20250815 02:08:36.145814 7021 consensus_queue.cc:260] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }
I20250815 02:08:36.146062 7021 ts_tablet_manager.cc:1428] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:36.146065 6613 heartbeater.cc:499] Master 127.6.59.190:41647 was elected leader, sending a full tablet report...
I20250815 02:08:36.146875 7022 raft_consensus.cc:357] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }
I20250815 02:08:36.146992 7022 raft_consensus.cc:383] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:36.147027 7022 raft_consensus.cc:738] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcfeb2efeec043948c8707b177881f0a, State: Initialized, Role: FOLLOWER
I20250815 02:08:36.147006 7026 tablet_bootstrap.cc:492] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8: Bootstrap starting.
I20250815 02:08:36.147110 7022 consensus_queue.cc:260] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }
I20250815 02:08:36.147353 7022 ts_tablet_manager.cc:1428] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20250815 02:08:36.147385 6744 heartbeater.cc:499] Master 127.6.59.190:41647 was elected leader, sending a full tablet report...
I20250815 02:08:36.147598 7026 tablet_bootstrap.cc:654] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:36.147879 7026 log.cc:826] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:36.148419 7026 tablet_bootstrap.cc:492] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8: No bootstrap required, opened a new log
I20250815 02:08:36.148474 7026 ts_tablet_manager.cc:1397] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:36.149431 7026 raft_consensus.cc:357] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }
I20250815 02:08:36.149537 7026 raft_consensus.cc:383] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:36.149571 7026 raft_consensus.cc:738] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bba4fd6eb6864cf182fd2fc34cbcd5a8, State: Initialized, Role: FOLLOWER
I20250815 02:08:36.149665 7026 consensus_queue.cc:260] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }
I20250815 02:08:36.149859 7026 ts_tablet_manager.cc:1428] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:36.149909 6875 heartbeater.cc:499] Master 127.6.59.190:41647 was elected leader, sending a full tablet report...
W20250815 02:08:36.254431 6614 tablet.cc:2378] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250815 02:08:36.254782 7027 raft_consensus.cc:491] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250815 02:08:36.254866 7027 raft_consensus.cc:513] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }
I20250815 02:08:36.255146 7027 leader_election.cc:290] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bba4fd6eb6864cf182fd2fc34cbcd5a8 (127.6.59.131:34477), 7ac0377c4adb40159b3c9ba9c1e2437b (127.6.59.129:42099)
W20250815 02:08:36.256247 6876 tablet.cc:2378] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250815 02:08:36.257547 6829 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f188c5ac9b434346869e614cd13ce64d" candidate_uuid: "bcfeb2efeec043948c8707b177881f0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" is_pre_election: true
I20250815 02:08:36.257705 6829 raft_consensus.cc:2466] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcfeb2efeec043948c8707b177881f0a in term 0.
I20250815 02:08:36.257929 6631 leader_election.cc:304] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bba4fd6eb6864cf182fd2fc34cbcd5a8, bcfeb2efeec043948c8707b177881f0a; no voters:
I20250815 02:08:36.258090 7027 raft_consensus.cc:2802] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250815 02:08:36.258145 7027 raft_consensus.cc:491] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250815 02:08:36.258235 7027 raft_consensus.cc:3058] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:36.258252 6567 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f188c5ac9b434346869e614cd13ce64d" candidate_uuid: "bcfeb2efeec043948c8707b177881f0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" is_pre_election: true
I20250815 02:08:36.258379 6567 raft_consensus.cc:2466] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcfeb2efeec043948c8707b177881f0a in term 0.
I20250815 02:08:36.259111 7027 raft_consensus.cc:513] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }
I20250815 02:08:36.259246 7027 leader_election.cc:290] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [CANDIDATE]: Term 1 election: Requested vote from peers bba4fd6eb6864cf182fd2fc34cbcd5a8 (127.6.59.131:34477), 7ac0377c4adb40159b3c9ba9c1e2437b (127.6.59.129:42099)
I20250815 02:08:36.259433 6829 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f188c5ac9b434346869e614cd13ce64d" candidate_uuid: "bcfeb2efeec043948c8707b177881f0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8"
I20250815 02:08:36.259500 6829 raft_consensus.cc:3058] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:36.260090 6829 raft_consensus.cc:2466] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcfeb2efeec043948c8707b177881f0a in term 1.
I20250815 02:08:36.260214 6567 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f188c5ac9b434346869e614cd13ce64d" candidate_uuid: "bcfeb2efeec043948c8707b177881f0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b"
I20250815 02:08:36.260264 6631 leader_election.cc:304] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bba4fd6eb6864cf182fd2fc34cbcd5a8, bcfeb2efeec043948c8707b177881f0a; no voters:
I20250815 02:08:36.260310 6567 raft_consensus.cc:3058] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:36.260382 7027 raft_consensus.cc:2802] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:36.260524 7027 raft_consensus.cc:695] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [term 1 LEADER]: Becoming Leader. State: Replica: bcfeb2efeec043948c8707b177881f0a, State: Running, Role: LEADER
I20250815 02:08:36.260617 7027 consensus_queue.cc:237] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }
I20250815 02:08:36.261060 6567 raft_consensus.cc:2466] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcfeb2efeec043948c8707b177881f0a in term 1.
I20250815 02:08:36.261379 6424 catalog_manager.cc:5582] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a reported cstate change: term changed from 0 to 1, leader changed from <none> to bcfeb2efeec043948c8707b177881f0a (127.6.59.130). New cstate: current_term: 1 leader_uuid: "bcfeb2efeec043948c8707b177881f0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } health_report { overall_health: UNKNOWN } } }
I20250815 02:08:36.282073 6382 external_mini_cluster.cc:949] 4 TS(s) registered with all masters
I20250815 02:08:36.282615 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 7ac0377c4adb40159b3c9ba9c1e2437b to finish bootstrapping
I20250815 02:08:36.286227 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver bcfeb2efeec043948c8707b177881f0a to finish bootstrapping
I20250815 02:08:36.289237 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver bba4fd6eb6864cf182fd2fc34cbcd5a8 to finish bootstrapping
I20250815 02:08:36.292392 6382 kudu-admin-test.cc:709] Waiting for Master to see the current replicas...
I20250815 02:08:36.292894 6382 kudu-admin-test.cc:716] Tablet locations:
tablet_locations {
tablet_id: "f188c5ac9b434346869e614cd13ce64d"
DEPRECATED_stale: false
partition {
partition_key_start: ""
partition_key_end: ""
}
interned_replicas {
ts_info_idx: 0
role: LEADER
}
interned_replicas {
ts_info_idx: 1
role: FOLLOWER
}
interned_replicas {
ts_info_idx: 2
role: FOLLOWER
}
}
ts_infos {
permanent_uuid: "bcfeb2efeec043948c8707b177881f0a"
rpc_addresses {
host: "127.6.59.130"
port: 36313
}
}
ts_infos {
permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8"
rpc_addresses {
host: "127.6.59.131"
port: 34477
}
}
ts_infos {
permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b"
rpc_addresses {
host: "127.6.59.129"
port: 42099
}
}
W20250815 02:08:36.378141 6745 tablet.cc:2378] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250815 02:08:36.706379 7027 consensus_queue.cc:1035] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:36.708395 7027 consensus_queue.cc:1035] T f188c5ac9b434346869e614cd13ce64d P bcfeb2efeec043948c8707b177881f0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:36.714874 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 6616
I20250815 02:08:36.719945 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 6394
I20250815 02:08:36.726711 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:41647
--webserver_interface=127.6.59.190
--webserver_port=45173
--builtin_ntp_servers=127.6.59.148:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:41647 with env {}
W20250815 02:08:36.804075 7043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:36.804244 7043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:36.804263 7043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:36.805604 7043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:36.805644 7043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:36.805657 7043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:36.805675 7043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:36.807130 7043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:41647
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:41647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=45173
--never_fsync=true
--heap_profile_path=/tmp/kudu.7043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:36.807328 7043 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:36.807502 7043 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:36.809702 7048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:36.809715 7051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:36.809720 7049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:36.809832 7043 server_base.cc:1047] running on GCE node
I20250815 02:08:36.810115 7043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:36.810352 7043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:36.811470 7043 hybrid_clock.cc:648] HybridClock initialized: now 1755223716811464 us; error 27 us; skew 500 ppm
I20250815 02:08:36.812577 7043 webserver.cc:480] Webserver started at http://127.6.59.190:45173/ using document root <none> and password file <none>
I20250815 02:08:36.812790 7043 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:36.812851 7043 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:36.814034 7043 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20250815 02:08:36.814664 7057 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:36.814831 7043 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:36.814898 7043 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
uuid: "2478edf97ac04278bc86d7ed81148566"
format_stamp: "Formatted at 2025-08-15 02:08:35 on dist-test-slave-wq2q"
I20250815 02:08:36.815141 7043 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:36.833222 7043 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:36.833529 7043 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:36.833652 7043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:36.837620 7043 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:41647
I20250815 02:08:36.837699 7109 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:41647 every 8 connection(s)
I20250815 02:08:36.838043 7043 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250815 02:08:36.839273 7110 sys_catalog.cc:263] Verifying existing consensus state
I20250815 02:08:36.839784 7110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566: Bootstrap starting.
I20250815 02:08:36.840761 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 7043
I20250815 02:08:36.840859 6382 kudu-admin-test.cc:735] Forcing unsafe config change on tserver bba4fd6eb6864cf182fd2fc34cbcd5a8
I20250815 02:08:36.847468 7110 log.cc:826] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:36.849120 7110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:36.849381 7110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566: Bootstrap complete.
I20250815 02:08:36.851358 7110 raft_consensus.cc:357] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } }
I20250815 02:08:36.851541 7110 raft_consensus.cc:738] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2478edf97ac04278bc86d7ed81148566, State: Initialized, Role: FOLLOWER
I20250815 02:08:36.851665 7110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } }
I20250815 02:08:36.851734 7110 raft_consensus.cc:397] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:36.851783 7110 raft_consensus.cc:491] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:36.851832 7110 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:36.852644 7110 raft_consensus.cc:513] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } }
I20250815 02:08:36.852731 7110 leader_election.cc:304] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2478edf97ac04278bc86d7ed81148566; no voters:
I20250815 02:08:36.852896 7110 leader_election.cc:290] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [CANDIDATE]: Term 2 election: Requested vote from peers
I20250815 02:08:36.852946 7114 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 2 FOLLOWER]: Leader election won for term 2
I20250815 02:08:36.853063 7114 raft_consensus.cc:695] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [term 2 LEADER]: Becoming Leader. State: Replica: 2478edf97ac04278bc86d7ed81148566, State: Running, Role: LEADER
I20250815 02:08:36.853158 7110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:36.853152 7114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } }
I20250815 02:08:36.853718 7114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2478edf97ac04278bc86d7ed81148566. Latest consensus state: current_term: 2 leader_uuid: "2478edf97ac04278bc86d7ed81148566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } } }
I20250815 02:08:36.853725 7116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2478edf97ac04278bc86d7ed81148566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2478edf97ac04278bc86d7ed81148566" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 41647 } } }
I20250815 02:08:36.853812 7114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:36.853832 7116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566 [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:36.854001 7121 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:36.854563 7121 catalog_manager.cc:671] Loaded metadata for table TestTable [id=e00ac1febb664a9faebe1eda59f05352]
I20250815 02:08:36.854805 7121 tablet_loader.cc:96] loaded metadata for tablet f188c5ac9b434346869e614cd13ce64d (table TestTable [id=e00ac1febb664a9faebe1eda59f05352])
I20250815 02:08:36.854863 7121 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:36.854961 7121 catalog_manager.cc:1261] Loaded cluster ID: 114ca840588b471989aa05ad6b1517d3
I20250815 02:08:36.854984 7121 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:36.856314 7121 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:36.856482 7121 catalog_manager.cc:5966] T 00000000000000000000000000000000 P 2478edf97ac04278bc86d7ed81148566: Loaded TSK: 0
I20250815 02:08:36.856676 7121 catalog_manager.cc:1516] Initializing in-progress tserver states...
W20250815 02:08:36.916911 7112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:36.917086 7112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:36.918531 7112 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20250815 02:08:36.930217 6829 tablet_service.cc:1905] Received UnsafeChangeConfig RPC: dest_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8"
tablet_id: "f188c5ac9b434346869e614cd13ce64d"
caller_id: "kudu-tools"
new_config {
peers {
permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b"
}
peers {
permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8"
}
}
from {username='slave'} at 127.0.0.1:44964
W20250815 02:08:36.930433 6829 raft_consensus.cc:2216] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 1 FOLLOWER]: PROCEEDING WITH UNSAFE CONFIG CHANGE ON THIS SERVER, COMMITTED CONFIG: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }NEW CONFIG: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } unsafe_config_change: true
I20250815 02:08:36.930542 6829 raft_consensus.cc:3058] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:36.935441 7074 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" instance_seqno: 1755223716004092) as {username='slave'} at 127.6.59.131:57715; Asking this server to re-register.
I20250815 02:08:36.935817 6875 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:36.935882 6875 heartbeater.cc:507] Master 127.6.59.190:41647 requested a full tablet report, sending...
I20250815 02:08:36.936188 7074 ts_manager.cc:194] Registered new tserver with Master: bba4fd6eb6864cf182fd2fc34cbcd5a8 (127.6.59.131:34477)
I20250815 02:08:37.128176 7007 heartbeater.cc:499] Master 127.6.59.190:41647 was elected leader, sending a full tablet report...
I20250815 02:08:37.131126 7074 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" instance_seqno: 1755223716119200) as {username='slave'} at 127.6.59.132:37847; Asking this server to re-register.
I20250815 02:08:37.131397 7007 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:37.131469 7007 heartbeater.cc:507] Master 127.6.59.190:41647 requested a full tablet report, sending...
I20250815 02:08:37.131695 7074 ts_manager.cc:194] Registered new tserver with Master: d53365830b6a4d2694d4abd1f91cc22c (127.6.59.132:36145)
I20250815 02:08:37.711148 7074 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" instance_seqno: 1755223715751581) as {username='slave'} at 127.6.59.129:45623; Asking this server to re-register.
I20250815 02:08:37.711412 6613 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:37.711491 6613 heartbeater.cc:507] Master 127.6.59.190:41647 requested a full tablet report, sending...
I20250815 02:08:37.711720 7074 ts_manager.cc:194] Registered new tserver with Master: 7ac0377c4adb40159b3c9ba9c1e2437b (127.6.59.129:42099)
I20250815 02:08:38.208333 7154 raft_consensus.cc:491] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcfeb2efeec043948c8707b177881f0a)
I20250815 02:08:38.208433 7154 raft_consensus.cc:513] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } }
I20250815 02:08:38.208737 7154 leader_election.cc:290] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcfeb2efeec043948c8707b177881f0a (127.6.59.130:36313), bba4fd6eb6864cf182fd2fc34cbcd5a8 (127.6.59.131:34477)
W20250815 02:08:38.209053 6503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.59.130:36313: connect: Connection refused (error 111)
W20250815 02:08:38.209471 6503 leader_election.cc:336] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bcfeb2efeec043948c8707b177881f0a (127.6.59.130:36313): Network error: Client connection negotiation failed: client connection to 127.6.59.130:36313: connect: Connection refused (error 111)
I20250815 02:08:38.211256 6829 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f188c5ac9b434346869e614cd13ce64d" candidate_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" is_pre_election: true
I20250815 02:08:38.211519 6500 leader_election.cc:304] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ac0377c4adb40159b3c9ba9c1e2437b; no voters: bba4fd6eb6864cf182fd2fc34cbcd5a8, bcfeb2efeec043948c8707b177881f0a
I20250815 02:08:38.211628 7154 raft_consensus.cc:2747] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250815 02:08:38.432305 7158 raft_consensus.cc:491] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader kudu-tools)
I20250815 02:08:38.432417 7158 raft_consensus.cc:513] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } unsafe_config_change: true
I20250815 02:08:38.432715 7158 leader_election.cc:290] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7ac0377c4adb40159b3c9ba9c1e2437b (127.6.59.129:42099)
I20250815 02:08:38.435339 6567 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f188c5ac9b434346869e614cd13ce64d" candidate_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" is_pre_election: true
I20250815 02:08:38.435431 6567 raft_consensus.cc:2466] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bba4fd6eb6864cf182fd2fc34cbcd5a8 in term 1.
I20250815 02:08:38.435618 6764 leader_election.cc:304] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 7ac0377c4adb40159b3c9ba9c1e2437b, bba4fd6eb6864cf182fd2fc34cbcd5a8; no voters:
I20250815 02:08:38.435738 7158 raft_consensus.cc:2802] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250815 02:08:38.435791 7158 raft_consensus.cc:491] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader kudu-tools)
I20250815 02:08:38.435806 7158 raft_consensus.cc:3058] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 2 FOLLOWER]: Advancing to term 3
I20250815 02:08:38.436391 7158 raft_consensus.cc:513] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } unsafe_config_change: true
I20250815 02:08:38.436481 7158 leader_election.cc:290] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [CANDIDATE]: Term 3 election: Requested vote from peers 7ac0377c4adb40159b3c9ba9c1e2437b (127.6.59.129:42099)
I20250815 02:08:38.436657 6567 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f188c5ac9b434346869e614cd13ce64d" candidate_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b"
I20250815 02:08:38.436723 6567 raft_consensus.cc:3058] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 1 FOLLOWER]: Advancing to term 3
I20250815 02:08:38.437223 6567 raft_consensus.cc:2466] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bba4fd6eb6864cf182fd2fc34cbcd5a8 in term 3.
I20250815 02:08:38.437376 6764 leader_election.cc:304] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 7ac0377c4adb40159b3c9ba9c1e2437b, bba4fd6eb6864cf182fd2fc34cbcd5a8; no voters:
I20250815 02:08:38.437469 7158 raft_consensus.cc:2802] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 3 FOLLOWER]: Leader election won for term 3
I20250815 02:08:38.437566 7158 raft_consensus.cc:695] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 3 LEADER]: Becoming Leader. State: Replica: bba4fd6eb6864cf182fd2fc34cbcd5a8, State: Running, Role: LEADER
I20250815 02:08:38.437640 7158 consensus_queue.cc:237] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } unsafe_config_change: true
I20250815 02:08:38.438314 7074 catalog_manager.cc:5582] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 reported cstate change: term changed from 1 to 3, leader changed from bcfeb2efeec043948c8707b177881f0a (127.6.59.130) to bba4fd6eb6864cf182fd2fc34cbcd5a8 (127.6.59.131), now has a pending config: VOTER bba4fd6eb6864cf182fd2fc34cbcd5a8 (127.6.59.131), VOTER 7ac0377c4adb40159b3c9ba9c1e2437b (127.6.59.129). New cstate: current_term: 3 leader_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcfeb2efeec043948c8707b177881f0a" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 36313 } } peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } unsafe_config_change: true }
I20250815 02:08:38.818323 6567 raft_consensus.cc:1273] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 3 FOLLOWER]: Refusing update from remote peer bba4fd6eb6864cf182fd2fc34cbcd5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250815 02:08:38.818660 7158 consensus_queue.cc:1035] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20250815 02:08:38.819842 7161 raft_consensus.cc:2953] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 3 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, VOTER bcfeb2efeec043948c8707b177881f0a (127.6.59.130) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } unsafe_config_change: true }
I20250815 02:08:38.819968 6567 raft_consensus.cc:2953] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 3 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, VOTER bcfeb2efeec043948c8707b177881f0a (127.6.59.130) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } unsafe_config_change: true }
I20250815 02:08:38.821122 7074 catalog_manager.cc:5582] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 reported cstate change: config changed from index -1 to 2, VOTER bcfeb2efeec043948c8707b177881f0a (127.6.59.130) evicted, no longer has a pending config: VOTER bba4fd6eb6864cf182fd2fc34cbcd5a8 (127.6.59.131), VOTER 7ac0377c4adb40159b3c9ba9c1e2437b (127.6.59.129). New cstate: current_term: 3 leader_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } health_report { overall_health: HEALTHY } } unsafe_config_change: true }
W20250815 02:08:38.821847 7074 catalog_manager.cc:5774] Failed to send DeleteTablet RPC for tablet f188c5ac9b434346869e614cd13ce64d on TS bcfeb2efeec043948c8707b177881f0a: Not found: failed to reset TS proxy: Could not find TS for UUID bcfeb2efeec043948c8707b177881f0a
I20250815 02:08:38.825040 6829 consensus_queue.cc:237] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: NON_VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: true } } unsafe_config_change: true
I20250815 02:08:38.825697 6567 raft_consensus.cc:1273] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 3 FOLLOWER]: Refusing update from remote peer bba4fd6eb6864cf182fd2fc34cbcd5a8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250815 02:08:38.826026 7161 consensus_queue.cc:1035] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250815 02:08:38.826668 7158 raft_consensus.cc:2953] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, NON_VOTER d53365830b6a4d2694d4abd1f91cc22c (127.6.59.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: NON_VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: true } } unsafe_config_change: true }
I20250815 02:08:38.826810 6567 raft_consensus.cc:2953] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, NON_VOTER d53365830b6a4d2694d4abd1f91cc22c (127.6.59.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: NON_VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: true } } unsafe_config_change: true }
I20250815 02:08:38.827831 7058 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f188c5ac9b434346869e614cd13ce64d with cas_config_opid_index 2: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20250815 02:08:38.827937 7074 catalog_manager.cc:5582] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b reported cstate change: config changed from index 2 to 4, NON_VOTER d53365830b6a4d2694d4abd1f91cc22c (127.6.59.132) added. New cstate: current_term: 3 leader_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: NON_VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: true } } unsafe_config_change: true }
W20250815 02:08:38.828665 6765 consensus_peers.cc:489] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 -> Peer d53365830b6a4d2694d4abd1f91cc22c (127.6.59.132:36145): Couldn't send request to peer d53365830b6a4d2694d4abd1f91cc22c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f188c5ac9b434346869e614cd13ce64d. This is attempt 1: this message will repeat every 5th retry.
W20250815 02:08:38.874929 7059 catalog_manager.cc:4726] Async tablet task DeleteTablet RPC for tablet f188c5ac9b434346869e614cd13ce64d on TS bcfeb2efeec043948c8707b177881f0a failed: Not found: failed to reset TS proxy: Could not find TS for UUID bcfeb2efeec043948c8707b177881f0a
I20250815 02:08:39.435593 7176 ts_tablet_manager.cc:927] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: Initiating tablet copy from peer bba4fd6eb6864cf182fd2fc34cbcd5a8 (127.6.59.131:34477)
I20250815 02:08:39.435837 7176 tablet_copy_client.cc:323] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: tablet copy: Beginning tablet copy session from remote peer at address 127.6.59.131:34477
I20250815 02:08:39.438570 6849 tablet_copy_service.cc:140] P bba4fd6eb6864cf182fd2fc34cbcd5a8: Received BeginTabletCopySession request for tablet f188c5ac9b434346869e614cd13ce64d from peer d53365830b6a4d2694d4abd1f91cc22c ({username='slave'} at 127.6.59.132:41495)
I20250815 02:08:39.438640 6849 tablet_copy_service.cc:161] P bba4fd6eb6864cf182fd2fc34cbcd5a8: Beginning new tablet copy session on tablet f188c5ac9b434346869e614cd13ce64d from peer d53365830b6a4d2694d4abd1f91cc22c at {username='slave'} at 127.6.59.132:41495: session id = d53365830b6a4d2694d4abd1f91cc22c-f188c5ac9b434346869e614cd13ce64d
I20250815 02:08:39.439131 6849 tablet_copy_source_session.cc:215] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8: Tablet Copy: opened 0 blocks and 1 log segments
I20250815 02:08:39.439888 7176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f188c5ac9b434346869e614cd13ce64d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:39.441767 7176 tablet_copy_client.cc:806] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: tablet copy: Starting download of 0 data blocks...
I20250815 02:08:39.441954 7176 tablet_copy_client.cc:670] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: tablet copy: Starting download of 1 WAL segments...
I20250815 02:08:39.442952 7176 tablet_copy_client.cc:538] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250815 02:08:39.443605 7176 tablet_bootstrap.cc:492] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: Bootstrap starting.
I20250815 02:08:39.444752 7176 log.cc:826] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:39.445638 7176 tablet_bootstrap.cc:492] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:39.445814 7176 tablet_bootstrap.cc:492] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: Bootstrap complete.
I20250815 02:08:39.445926 7176 ts_tablet_manager.cc:1397] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250815 02:08:39.446883 7176 raft_consensus.cc:357] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: NON_VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: true } } unsafe_config_change: true
I20250815 02:08:39.447011 7176 raft_consensus.cc:738] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: d53365830b6a4d2694d4abd1f91cc22c, State: Initialized, Role: LEARNER
I20250815 02:08:39.447081 7176 consensus_queue.cc:260] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: NON_VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: true } } unsafe_config_change: true
I20250815 02:08:39.447297 7176 ts_tablet_manager.cc:1428] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:39.447623 6849 tablet_copy_service.cc:342] P bba4fd6eb6864cf182fd2fc34cbcd5a8: Request end of tablet copy session d53365830b6a4d2694d4abd1f91cc22c-f188c5ac9b434346869e614cd13ce64d received from {username='slave'} at 127.6.59.132:41495
I20250815 02:08:39.447691 6849 tablet_copy_service.cc:434] P bba4fd6eb6864cf182fd2fc34cbcd5a8: ending tablet copy session d53365830b6a4d2694d4abd1f91cc22c-f188c5ac9b434346869e614cd13ce64d on tablet f188c5ac9b434346869e614cd13ce64d with peer d53365830b6a4d2694d4abd1f91cc22c
I20250815 02:08:40.013914 6961 raft_consensus.cc:1215] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4] Dedup: 3.4->[]
W20250815 02:08:40.039794 7059 catalog_manager.cc:4726] Async tablet task DeleteTablet RPC for tablet f188c5ac9b434346869e614cd13ce64d on TS bcfeb2efeec043948c8707b177881f0a failed: Not found: failed to reset TS proxy: Could not find TS for UUID bcfeb2efeec043948c8707b177881f0a
I20250815 02:08:40.417050 7183 raft_consensus.cc:1062] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8: attempting to promote NON_VOTER d53365830b6a4d2694d4abd1f91cc22c to VOTER
I20250815 02:08:40.417285 7183 consensus_queue.cc:237] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: false } } unsafe_config_change: true
I20250815 02:08:40.417790 6961 raft_consensus.cc:1273] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c [term 3 LEARNER]: Refusing update from remote peer bba4fd6eb6864cf182fd2fc34cbcd5a8: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20250815 02:08:40.417814 6567 raft_consensus.cc:1273] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 3 FOLLOWER]: Refusing update from remote peer bba4fd6eb6864cf182fd2fc34cbcd5a8: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20250815 02:08:40.418051 7183 consensus_queue.cc:1035] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250815 02:08:40.418172 7186 consensus_queue.cc:1035] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250815 02:08:40.418963 7186 raft_consensus.cc:2953] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, d53365830b6a4d2694d4abd1f91cc22c (127.6.59.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: false } } unsafe_config_change: true }
I20250815 02:08:40.419081 6567 raft_consensus.cc:2953] T f188c5ac9b434346869e614cd13ce64d P 7ac0377c4adb40159b3c9ba9c1e2437b [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, d53365830b6a4d2694d4abd1f91cc22c (127.6.59.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: false } } unsafe_config_change: true }
I20250815 02:08:40.420187 6961 raft_consensus.cc:2953] T f188c5ac9b434346869e614cd13ce64d P d53365830b6a4d2694d4abd1f91cc22c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, d53365830b6a4d2694d4abd1f91cc22c (127.6.59.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: false } } unsafe_config_change: true }
I20250815 02:08:40.420297 7074 catalog_manager.cc:5582] T f188c5ac9b434346869e614cd13ce64d P bba4fd6eb6864cf182fd2fc34cbcd5a8 reported cstate change: config changed from index 4 to 5, d53365830b6a4d2694d4abd1f91cc22c (127.6.59.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 34477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 42099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c" member_type: VOTER last_known_addr { host: "127.6.59.132" port: 36145 } attrs { promote: false } health_report { overall_health: HEALTHY } } unsafe_config_change: true }
I20250815 02:08:40.469303 6382 kudu-admin-test.cc:751] Waiting for Master to see new config...
I20250815 02:08:40.472822 6382 kudu-admin-test.cc:756] Tablet locations:
tablet_locations {
tablet_id: "f188c5ac9b434346869e614cd13ce64d"
DEPRECATED_stale: false
partition {
partition_key_start: ""
partition_key_end: ""
}
interned_replicas {
ts_info_idx: 0
role: LEADER
}
interned_replicas {
ts_info_idx: 1
role: FOLLOWER
}
interned_replicas {
ts_info_idx: 2
role: FOLLOWER
}
}
ts_infos {
permanent_uuid: "bba4fd6eb6864cf182fd2fc34cbcd5a8"
rpc_addresses {
host: "127.6.59.131"
port: 34477
}
}
ts_infos {
permanent_uuid: "7ac0377c4adb40159b3c9ba9c1e2437b"
rpc_addresses {
host: "127.6.59.129"
port: 42099
}
}
ts_infos {
permanent_uuid: "d53365830b6a4d2694d4abd1f91cc22c"
rpc_addresses {
host: "127.6.59.132"
port: 36145
}
}
I20250815 02:08:40.473246 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 6485
I20250815 02:08:40.478714 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 6747
I20250815 02:08:40.482882 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 6878
I20250815 02:08:40.486774 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 7043
2025-08-15T02:08:40Z chronyd exiting
[ OK ] AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes (5039 ms)
[ RUN ] AdminCliTest.TestGracefulSpecificLeaderStepDown
I20250815 02:08:40.507421 6382 test_util.cc:276] Using random seed: -564348902
I20250815 02:08:40.507882 6382 ts_itest-base.cc:115] Starting cluster with:
I20250815 02:08:40.507920 6382 ts_itest-base.cc:116] --------------
I20250815 02:08:40.507930 6382 ts_itest-base.cc:117] 3 tablet servers
I20250815 02:08:40.507941 6382 ts_itest-base.cc:118] 3 replicas per TS
I20250815 02:08:40.507951 6382 ts_itest-base.cc:119] --------------
2025-08-15T02:08:40Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-15T02:08:40Z Disabled control of system clock
I20250815 02:08:40.514354 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:35019
--webserver_interface=127.6.59.190
--webserver_port=0
--builtin_ntp_servers=127.6.59.148:44431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:35019
--catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {}
W20250815 02:08:40.589951 7203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:40.590183 7203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:40.590212 7203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:40.591588 7203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:40.591631 7203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:40.591660 7203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:40.591678 7203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:40.593168 7203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:44431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:35019
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:35019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:40.593408 7203 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:40.593637 7203 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:40.595986 7209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:40.596050 7208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:40.596079 7203 server_base.cc:1047] running on GCE node
W20250815 02:08:40.595986 7211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:40.596410 7203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:40.596614 7203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:40.597743 7203 hybrid_clock.cc:648] HybridClock initialized: now 1755223720597729 us; error 29 us; skew 500 ppm
I20250815 02:08:40.598889 7203 webserver.cc:480] Webserver started at http://127.6.59.190:46341/ using document root <none> and password file <none>
I20250815 02:08:40.599071 7203 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:40.599110 7203 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:40.599220 7203 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:40.600062 7203 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.600342 7203 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.601488 7203 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250815 02:08:40.602151 7217 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:40.602320 7203 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:40.602391 7203 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.602452 7203 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:40.611176 7203 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:40.611402 7203 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:40.611507 7203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:40.615149 7203 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:35019
I20250815 02:08:40.615206 7269 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:35019 every 8 connection(s)
I20250815 02:08:40.615486 7203 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250815 02:08:40.615998 7270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:40.617664 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 7203
I20250815 02:08:40.617764 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250815 02:08:40.618350 7270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f: Bootstrap starting.
I20250815 02:08:40.618945 7270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:40.619185 7270 log.cc:826] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:40.619851 7270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f: No bootstrap required, opened a new log
I20250815 02:08:40.621078 7270 raft_consensus.cc:357] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35019 } }
I20250815 02:08:40.621193 7270 raft_consensus.cc:383] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:40.621227 7270 raft_consensus.cc:738] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2377dd457f4936b7d4d8f4dc6ce71f, State: Initialized, Role: FOLLOWER
I20250815 02:08:40.621389 7270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35019 } }
I20250815 02:08:40.621850 7270 raft_consensus.cc:397] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:40.621891 7270 raft_consensus.cc:491] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:40.621932 7270 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:40.622598 7270 raft_consensus.cc:513] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35019 } }
I20250815 02:08:40.622722 7270 leader_election.cc:304] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c2377dd457f4936b7d4d8f4dc6ce71f; no voters:
I20250815 02:08:40.622737 7275 raft_consensus.cc:491] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250815 02:08:40.622798 7275 raft_consensus.cc:513] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35019 } }
I20250815 02:08:40.622874 7270 leader_election.cc:290] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:40.623052 7275 leader_election.cc:304] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c2377dd457f4936b7d4d8f4dc6ce71f; no voters:
I20250815 02:08:40.623067 7277 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:40.623121 7270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:40.623219 7275 leader_election.cc:290] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers
I20250815 02:08:40.623265 7277 raft_consensus.cc:695] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [term 1 LEADER]: Becoming Leader. State: Replica: 1c2377dd457f4936b7d4d8f4dc6ce71f, State: Running, Role: LEADER
I20250815 02:08:40.623374 7277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35019 } }
I20250815 02:08:40.623682 7277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c2377dd457f4936b7d4d8f4dc6ce71f. Latest consensus state: current_term: 1 leader_uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35019 } } }
I20250815 02:08:40.623858 7277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:40.623719 7276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2377dd457f4936b7d4d8f4dc6ce71f" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35019 } } }
I20250815 02:08:40.624135 7276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:40.624248 7286 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:40.624683 7286 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:40.626109 7286 catalog_manager.cc:1349] Generated new cluster ID: 791283633189492caee4c44fc2de885d
I20250815 02:08:40.626152 7286 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:40.645367 7286 catalog_manager.cc:1372] Generated new certificate authority record
I20250815 02:08:40.645925 7286 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:40.651548 7286 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 1c2377dd457f4936b7d4d8f4dc6ce71f: Generated new TSK 0
I20250815 02:08:40.651710 7286 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250815 02:08:40.654227 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.129:0
--local_ip_for_outbound_sockets=127.6.59.129
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35019
--builtin_ntp_servers=127.6.59.148:44431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20250815 02:08:40.735873 7296 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20250815 02:08:40.736042 7296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:40.736057 7296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:40.736074 7296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:40.737447 7296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:40.737500 7296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.129
I20250815 02:08:40.738978 7296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:44431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35019
--never_fsync=true
--heap_profile_path=/tmp/kudu.7296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.129
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:40.739198 7296 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:40.739418 7296 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:40.742058 7304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:40.742085 7302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:40.742276 7296 server_base.cc:1047] running on GCE node
W20250815 02:08:40.742146 7301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:40.742501 7296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:40.742726 7296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:40.743873 7296 hybrid_clock.cc:648] HybridClock initialized: now 1755223720743864 us; error 33 us; skew 500 ppm
I20250815 02:08:40.744977 7296 webserver.cc:480] Webserver started at http://127.6.59.129:36455/ using document root <none> and password file <none>
I20250815 02:08:40.745206 7296 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:40.745255 7296 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:40.745373 7296 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:40.746258 7296 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "68f0a72bc176463eaa2b41df1cbc63e5"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.746542 7296 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "68f0a72bc176463eaa2b41df1cbc63e5"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.747713 7296 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:40.748481 7310 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:40.748677 7296 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:40.748751 7296 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "68f0a72bc176463eaa2b41df1cbc63e5"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.748813 7296 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:40.761884 7296 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:40.762142 7296 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:40.762264 7296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:40.762466 7296 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:40.762787 7296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:40.762820 7296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:40.762856 7296 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:40.762878 7296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:40.768393 7296 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.129:37911
I20250815 02:08:40.768453 7423 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.129:37911 every 8 connection(s)
I20250815 02:08:40.768815 7296 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250815 02:08:40.778781 7424 heartbeater.cc:344] Connected to a master server at 127.6.59.190:35019
I20250815 02:08:40.778825 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 7296
I20250815 02:08:40.778899 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250815 02:08:40.778910 7424 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:40.779139 7424 heartbeater.cc:507] Master 127.6.59.190:35019 requested a full tablet report, sending...
I20250815 02:08:40.779593 7233 ts_manager.cc:194] Registered new tserver with Master: 68f0a72bc176463eaa2b41df1cbc63e5 (127.6.59.129:37911)
I20250815 02:08:40.780055 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.130:0
--local_ip_for_outbound_sockets=127.6.59.130
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35019
--builtin_ntp_servers=127.6.59.148:44431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
I20250815 02:08:40.780371 7233 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.129:43251
W20250815 02:08:40.856160 7427 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20250815 02:08:40.856324 7427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:40.856339 7427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:40.856356 7427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:40.857625 7427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:40.857673 7427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.130
I20250815 02:08:40.859062 7427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:44431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35019
--never_fsync=true
--heap_profile_path=/tmp/kudu.7427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.130
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:40.859229 7427 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:40.859426 7427 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:40.861792 7432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:40.861781 7433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:40.862021 7427 server_base.cc:1047] running on GCE node
W20250815 02:08:40.861784 7435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:40.862296 7427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:40.862488 7427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:40.863636 7427 hybrid_clock.cc:648] HybridClock initialized: now 1755223720863615 us; error 33 us; skew 500 ppm
I20250815 02:08:40.864691 7427 webserver.cc:480] Webserver started at http://127.6.59.130:46001/ using document root <none> and password file <none>
I20250815 02:08:40.864876 7427 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:40.864920 7427 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:40.865032 7427 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:40.865841 7427 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.866144 7427 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.867225 7427 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250815 02:08:40.867838 7441 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:40.867987 7427 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:40.868055 7427 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.868111 7427 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:40.880733 7427 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:40.880954 7427 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:40.881062 7427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:40.881250 7427 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:40.881531 7427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:40.881562 7427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:40.881598 7427 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:40.881632 7427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:40.886986 7427 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.130:46431
I20250815 02:08:40.887058 7554 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.130:46431 every 8 connection(s)
I20250815 02:08:40.887322 7427 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250815 02:08:40.891453 7555 heartbeater.cc:344] Connected to a master server at 127.6.59.190:35019
I20250815 02:08:40.891534 7555 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:40.891744 7555 heartbeater.cc:507] Master 127.6.59.190:35019 requested a full tablet report, sending...
I20250815 02:08:40.892084 7233 ts_manager.cc:194] Registered new tserver with Master: 35b4da3fe32e4a8c8ea122f76c4c71d0 (127.6.59.130:46431)
I20250815 02:08:40.892431 7233 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.130:38821
I20250815 02:08:40.894145 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 7427
I20250815 02:08:40.894228 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250815 02:08:40.895346 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.131:0
--local_ip_for_outbound_sockets=127.6.59.131
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35019
--builtin_ntp_servers=127.6.59.148:44431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20250815 02:08:40.971143 7558 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20250815 02:08:40.971305 7558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:40.971328 7558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:40.971349 7558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:40.972721 7558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:40.972774 7558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.131
I20250815 02:08:40.974246 7558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:44431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35019
--never_fsync=true
--heap_profile_path=/tmp/kudu.7558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.131
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:40.974475 7558 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:40.974684 7558 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:40.977169 7564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:40.977195 7566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:40.977181 7558 server_base.cc:1047] running on GCE node
W20250815 02:08:40.977167 7563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:40.977485 7558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:40.977677 7558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:40.978808 7558 hybrid_clock.cc:648] HybridClock initialized: now 1755223720978790 us; error 31 us; skew 500 ppm
I20250815 02:08:40.979908 7558 webserver.cc:480] Webserver started at http://127.6.59.131:36013/ using document root <none> and password file <none>
I20250815 02:08:40.980099 7558 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:40.980142 7558 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:40.980250 7558 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:40.981050 7558 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "8c514bf32ccd49a399894aa471281016"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.981323 7558 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "8c514bf32ccd49a399894aa471281016"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.982440 7558 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:40.983070 7572 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:40.983222 7558 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:40.983306 7558 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "8c514bf32ccd49a399894aa471281016"
format_stamp: "Formatted at 2025-08-15 02:08:40 on dist-test-slave-wq2q"
I20250815 02:08:40.983361 7558 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:41.002233 7558 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:41.002460 7558 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:41.002562 7558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:41.002760 7558 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:41.003054 7558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:41.003087 7558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:41.003123 7558 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:41.003151 7558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:41.008591 7558 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.131:43447
I20250815 02:08:41.008687 7685 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.131:43447 every 8 connection(s)
I20250815 02:08:41.008966 7558 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250815 02:08:41.010157 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 7558
I20250815 02:08:41.010232 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250815 02:08:41.014101 7686 heartbeater.cc:344] Connected to a master server at 127.6.59.190:35019
I20250815 02:08:41.014184 7686 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:41.014360 7686 heartbeater.cc:507] Master 127.6.59.190:35019 requested a full tablet report, sending...
I20250815 02:08:41.014689 7233 ts_manager.cc:194] Registered new tserver with Master: 8c514bf32ccd49a399894aa471281016 (127.6.59.131:43447)
I20250815 02:08:41.015019 7233 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.131:47139
I20250815 02:08:41.022018 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:41.027204 7233 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51070:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20250815 02:08:41.028432 7233 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250815 02:08:41.034296 7489 tablet_service.cc:1468] Processing CreateTablet for tablet 68c15feee8574c8fa6e3f67f3a8ecdbf (DEFAULT_TABLE table=TestTable [id=b259ce494b2840cbb430666082d67a30]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:41.034622 7489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68c15feee8574c8fa6e3f67f3a8ecdbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:41.035063 7358 tablet_service.cc:1468] Processing CreateTablet for tablet 68c15feee8574c8fa6e3f67f3a8ecdbf (DEFAULT_TABLE table=TestTable [id=b259ce494b2840cbb430666082d67a30]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:41.035317 7358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68c15feee8574c8fa6e3f67f3a8ecdbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:41.036922 7700 tablet_bootstrap.cc:492] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0: Bootstrap starting.
I20250815 02:08:41.037314 7701 tablet_bootstrap.cc:492] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5: Bootstrap starting.
I20250815 02:08:41.037635 7700 tablet_bootstrap.cc:654] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:41.037890 7700 log.cc:826] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:41.037793 7620 tablet_service.cc:1468] Processing CreateTablet for tablet 68c15feee8574c8fa6e3f67f3a8ecdbf (DEFAULT_TABLE table=TestTable [id=b259ce494b2840cbb430666082d67a30]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:41.037902 7701 tablet_bootstrap.cc:654] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:41.038091 7620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68c15feee8574c8fa6e3f67f3a8ecdbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:41.038213 7701 log.cc:826] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:41.038635 7700 tablet_bootstrap.cc:492] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0: No bootstrap required, opened a new log
I20250815 02:08:41.038705 7700 ts_tablet_manager.cc:1397] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:41.038813 7701 tablet_bootstrap.cc:492] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5: No bootstrap required, opened a new log
I20250815 02:08:41.038879 7701 ts_tablet_manager.cc:1397] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:41.039779 7700 raft_consensus.cc:357] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:41.039901 7700 raft_consensus.cc:738] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b4da3fe32e4a8c8ea122f76c4c71d0, State: Initialized, Role: FOLLOWER
I20250815 02:08:41.039959 7700 consensus_queue.cc:260] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:41.040228 7700 ts_tablet_manager.cc:1428] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:41.040243 7555 heartbeater.cc:499] Master 127.6.59.190:35019 was elected leader, sending a full tablet report...
I20250815 02:08:41.040441 7701 raft_consensus.cc:357] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:41.040557 7701 raft_consensus.cc:738] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f0a72bc176463eaa2b41df1cbc63e5, State: Initialized, Role: FOLLOWER
I20250815 02:08:41.040557 7705 tablet_bootstrap.cc:492] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016: Bootstrap starting.
I20250815 02:08:41.040607 7701 consensus_queue.cc:260] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:41.040835 7701 ts_tablet_manager.cc:1428] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250815 02:08:41.040896 7424 heartbeater.cc:499] Master 127.6.59.190:35019 was elected leader, sending a full tablet report...
I20250815 02:08:41.041555 7705 tablet_bootstrap.cc:654] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:41.041848 7705 log.cc:826] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:41.042539 7705 tablet_bootstrap.cc:492] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016: No bootstrap required, opened a new log
I20250815 02:08:41.042599 7705 ts_tablet_manager.cc:1397] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250815 02:08:41.043004 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:41.043373 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 68f0a72bc176463eaa2b41df1cbc63e5 to finish bootstrapping
I20250815 02:08:41.043869 7705 raft_consensus.cc:357] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:41.044008 7705 raft_consensus.cc:738] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c514bf32ccd49a399894aa471281016, State: Initialized, Role: FOLLOWER
I20250815 02:08:41.044059 7705 consensus_queue.cc:260] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:41.044265 7705 ts_tablet_manager.cc:1428] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250815 02:08:41.044309 7686 heartbeater.cc:499] Master 127.6.59.190:35019 was elected leader, sending a full tablet report...
I20250815 02:08:41.046800 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 35b4da3fe32e4a8c8ea122f76c4c71d0 to finish bootstrapping
I20250815 02:08:41.049546 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 8c514bf32ccd49a399894aa471281016 to finish bootstrapping
I20250815 02:08:41.059834 7509 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "68c15feee8574c8fa6e3f67f3a8ecdbf"
dest_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0"
from {username='slave'} at 127.0.0.1:48292
I20250815 02:08:41.059937 7509 raft_consensus.cc:491] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20250815 02:08:41.059989 7509 raft_consensus.cc:3058] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:41.060619 7509 raft_consensus.cc:513] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:41.060923 7509 leader_election.cc:290] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [CANDIDATE]: Term 1 election: Requested vote from peers 8c514bf32ccd49a399894aa471281016 (127.6.59.131:43447), 68f0a72bc176463eaa2b41df1cbc63e5 (127.6.59.129:37911)
I20250815 02:08:41.063162 6382 cluster_itest_util.cc:257] Not converged past 1 yet: 0.0 0.0 0.0
I20250815 02:08:41.063481 7640 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "68c15feee8574c8fa6e3f67f3a8ecdbf" candidate_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8c514bf32ccd49a399894aa471281016"
I20250815 02:08:41.063565 7640 raft_consensus.cc:3058] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:41.064232 7640 raft_consensus.cc:2466] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b4da3fe32e4a8c8ea122f76c4c71d0 in term 1.
I20250815 02:08:41.064260 7378 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "68c15feee8574c8fa6e3f67f3a8ecdbf" candidate_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "68f0a72bc176463eaa2b41df1cbc63e5"
I20250815 02:08:41.064352 7378 raft_consensus.cc:3058] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:41.064437 7442 leader_election.cc:304] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b4da3fe32e4a8c8ea122f76c4c71d0, 8c514bf32ccd49a399894aa471281016; no voters:
I20250815 02:08:41.064564 7704 raft_consensus.cc:2802] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:41.064707 7704 raft_consensus.cc:695] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 1 LEADER]: Becoming Leader. State: Replica: 35b4da3fe32e4a8c8ea122f76c4c71d0, State: Running, Role: LEADER
I20250815 02:08:41.064805 7704 consensus_queue.cc:237] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:41.065446 7378 raft_consensus.cc:2466] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b4da3fe32e4a8c8ea122f76c4c71d0 in term 1.
I20250815 02:08:41.065694 7234 catalog_manager.cc:5582] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35b4da3fe32e4a8c8ea122f76c4c71d0 (127.6.59.130). New cstate: current_term: 1 leader_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } health_report { overall_health: UNKNOWN } } }
W20250815 02:08:41.137810 7556 tablet.cc:2378] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250815 02:08:41.164729 6382 cluster_itest_util.cc:257] Not converged past 1 yet: 1.1 0.0 0.0
W20250815 02:08:41.259521 7687 tablet.cc:2378] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250815 02:08:41.269440 7425 tablet.cc:2378] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250815 02:08:41.366165 6382 cluster_itest_util.cc:257] Not converged past 1 yet: 1.1 0.0 0.0
I20250815 02:08:41.596051 7713 consensus_queue.cc:1035] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:41.598148 7716 consensus_queue.cc:1035] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:41.757437 7509 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "68c15feee8574c8fa6e3f67f3a8ecdbf"
dest_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:48308
I20250815 02:08:41.757530 7509 raft_consensus.cc:604] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 1 LEADER]: Received request to transfer leadership
I20250815 02:08:42.134832 7740 raft_consensus.cc:991] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0: : Instructing follower 8c514bf32ccd49a399894aa471281016 to start an election
I20250815 02:08:42.134953 7740 raft_consensus.cc:1079] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 1 LEADER]: Signalling peer 8c514bf32ccd49a399894aa471281016 to start an election
I20250815 02:08:42.135170 7640 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "68c15feee8574c8fa6e3f67f3a8ecdbf"
dest_uuid: "8c514bf32ccd49a399894aa471281016"
from {username='slave'} at 127.6.59.130:42751
I20250815 02:08:42.135269 7640 raft_consensus.cc:491] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250815 02:08:42.135291 7640 raft_consensus.cc:3058] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:42.135865 7640 raft_consensus.cc:513] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:42.136111 7640 leader_election.cc:290] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [CANDIDATE]: Term 2 election: Requested vote from peers 35b4da3fe32e4a8c8ea122f76c4c71d0 (127.6.59.130:46431), 68f0a72bc176463eaa2b41df1cbc63e5 (127.6.59.129:37911)
I20250815 02:08:42.138676 7509 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "68c15feee8574c8fa6e3f67f3a8ecdbf" candidate_uuid: "8c514bf32ccd49a399894aa471281016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0"
I20250815 02:08:42.138765 7509 raft_consensus.cc:3053] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 1 LEADER]: Stepping down as leader of term 1
I20250815 02:08:42.138793 7509 raft_consensus.cc:738] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35b4da3fe32e4a8c8ea122f76c4c71d0, State: Running, Role: LEADER
I20250815 02:08:42.138763 7378 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "68c15feee8574c8fa6e3f67f3a8ecdbf" candidate_uuid: "8c514bf32ccd49a399894aa471281016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68f0a72bc176463eaa2b41df1cbc63e5"
I20250815 02:08:42.138859 7378 raft_consensus.cc:3058] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:42.138840 7509 consensus_queue.cc:260] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:42.138943 7509 raft_consensus.cc:3058] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:42.139448 7378 raft_consensus.cc:2466] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 68f0a72bc176463eaa2b41df1cbc63e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c514bf32ccd49a399894aa471281016 in term 2.
I20250815 02:08:42.139459 7509 raft_consensus.cc:2466] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c514bf32ccd49a399894aa471281016 in term 2.
I20250815 02:08:42.139653 7575 leader_election.cc:304] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f0a72bc176463eaa2b41df1cbc63e5, 8c514bf32ccd49a399894aa471281016; no voters:
I20250815 02:08:42.139817 7745 raft_consensus.cc:2802] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [term 2 FOLLOWER]: Leader election won for term 2
I20250815 02:08:42.139986 7745 raft_consensus.cc:695] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [term 2 LEADER]: Becoming Leader. State: Replica: 8c514bf32ccd49a399894aa471281016, State: Running, Role: LEADER
I20250815 02:08:42.140084 7745 consensus_queue.cc:237] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } }
I20250815 02:08:42.140707 7233 catalog_manager.cc:5582] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 8c514bf32ccd49a399894aa471281016 reported cstate change: term changed from 1 to 2, leader changed from 35b4da3fe32e4a8c8ea122f76c4c71d0 (127.6.59.130) to 8c514bf32ccd49a399894aa471281016 (127.6.59.131). New cstate: current_term: 2 leader_uuid: "8c514bf32ccd49a399894aa471281016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c514bf32ccd49a399894aa471281016" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 46431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68f0a72bc176463eaa2b41df1cbc63e5" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20250815 02:08:42.425129 7509 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "68c15feee8574c8fa6e3f67f3a8ecdbf"
dest_uuid: "35b4da3fe32e4a8c8ea122f76c4c71d0"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:48320
I20250815 02:08:42.425215 7509 raft_consensus.cc:604] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 2 FOLLOWER]: Received request to transfer leadership
I20250815 02:08:42.425243 7509 raft_consensus.cc:612] T 68c15feee8574c8fa6e3f67f3a8ecdbf P 35b4da3fe32e4a8c8ea122f76c4c71d0 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20250815 02:08:42.481061 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 7296
I20250815 02:08:42.487895 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 7427
I20250815 02:08:42.493358 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 7558
I20250815 02:08:42.498850 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 7203
2025-08-15T02:08:42Z chronyd exiting
[ OK ] AdminCliTest.TestGracefulSpecificLeaderStepDown (2011 ms)
[ RUN ] AdminCliTest.TestDescribeTableColumnFlags
I20250815 02:08:42.518590 6382 test_util.cc:276] Using random seed: -562337735
I20250815 02:08:42.519089 6382 ts_itest-base.cc:115] Starting cluster with:
I20250815 02:08:42.519129 6382 ts_itest-base.cc:116] --------------
I20250815 02:08:42.519148 6382 ts_itest-base.cc:117] 3 tablet servers
I20250815 02:08:42.519165 6382 ts_itest-base.cc:118] 3 replicas per TS
I20250815 02:08:42.519191 6382 ts_itest-base.cc:119] --------------
2025-08-15T02:08:42Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-15T02:08:42Z Disabled control of system clock
I20250815 02:08:42.525696 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:40299
--webserver_interface=127.6.59.190
--webserver_port=0
--builtin_ntp_servers=127.6.59.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:40299 with env {}
W20250815 02:08:42.602809 7774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:42.602967 7774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:42.602998 7774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:42.604313 7774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:42.604353 7774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:42.604374 7774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:42.604391 7774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:42.605858 7774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:40299
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:40299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:42.606109 7774 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:42.606317 7774 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:42.608685 7779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:42.608713 7780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:42.608840 7774 server_base.cc:1047] running on GCE node
W20250815 02:08:42.608733 7782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:42.609122 7774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:42.609349 7774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:42.610474 7774 hybrid_clock.cc:648] HybridClock initialized: now 1755223722610457 us; error 33 us; skew 500 ppm
I20250815 02:08:42.611570 7774 webserver.cc:480] Webserver started at http://127.6.59.190:45529/ using document root <none> and password file <none>
I20250815 02:08:42.611783 7774 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:42.611826 7774 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:42.611958 7774 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:42.612787 7774 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "488c3a0d671942e3bbead647c317bc0a"
format_stamp: "Formatted at 2025-08-15 02:08:42 on dist-test-slave-wq2q"
I20250815 02:08:42.613076 7774 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "488c3a0d671942e3bbead647c317bc0a"
format_stamp: "Formatted at 2025-08-15 02:08:42 on dist-test-slave-wq2q"
I20250815 02:08:42.614207 7774 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:42.614867 7788 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:42.615036 7774 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:42.615104 7774 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
uuid: "488c3a0d671942e3bbead647c317bc0a"
format_stamp: "Formatted at 2025-08-15 02:08:42 on dist-test-slave-wq2q"
I20250815 02:08:42.615168 7774 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:42.626968 7774 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:42.627228 7774 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:42.627336 7774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:42.630980 7774 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:40299
I20250815 02:08:42.631057 7840 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:40299 every 8 connection(s)
I20250815 02:08:42.631326 7774 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250815 02:08:42.631858 7841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:42.634171 7841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a: Bootstrap starting.
I20250815 02:08:42.634732 7841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:42.635002 7841 log.cc:826] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:42.635572 7841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a: No bootstrap required, opened a new log
I20250815 02:08:42.636655 7841 raft_consensus.cc:357] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488c3a0d671942e3bbead647c317bc0a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 40299 } }
I20250815 02:08:42.636771 7841 raft_consensus.cc:383] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:42.636792 7841 raft_consensus.cc:738] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 488c3a0d671942e3bbead647c317bc0a, State: Initialized, Role: FOLLOWER
I20250815 02:08:42.636945 7841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488c3a0d671942e3bbead647c317bc0a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 40299 } }
I20250815 02:08:42.637003 7841 raft_consensus.cc:397] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:42.637029 7841 raft_consensus.cc:491] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:42.637076 7841 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:42.637553 7841 raft_consensus.cc:513] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488c3a0d671942e3bbead647c317bc0a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 40299 } }
I20250815 02:08:42.637645 7841 leader_election.cc:304] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 488c3a0d671942e3bbead647c317bc0a; no voters:
I20250815 02:08:42.637781 7841 leader_election.cc:290] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:42.637848 7844 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:42.638020 7844 raft_consensus.cc:695] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [term 1 LEADER]: Becoming Leader. State: Replica: 488c3a0d671942e3bbead647c317bc0a, State: Running, Role: LEADER
I20250815 02:08:42.638060 7841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:42.638163 7844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488c3a0d671942e3bbead647c317bc0a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 40299 } }
I20250815 02:08:42.638499 7845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "488c3a0d671942e3bbead647c317bc0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488c3a0d671942e3bbead647c317bc0a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 40299 } } }
I20250815 02:08:42.638523 7846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 488c3a0d671942e3bbead647c317bc0a. Latest consensus state: current_term: 1 leader_uuid: "488c3a0d671942e3bbead647c317bc0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488c3a0d671942e3bbead647c317bc0a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 40299 } } }
I20250815 02:08:42.638623 7845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:42.638686 7846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:42.639012 7852 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:42.639434 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 7774
I20250815 02:08:42.639468 7852 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:42.639539 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250815 02:08:42.640899 7852 catalog_manager.cc:1349] Generated new cluster ID: 0bb48113705646c98cb671af571cf2d1
I20250815 02:08:42.640955 7852 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:42.680059 7852 catalog_manager.cc:1372] Generated new certificate authority record
I20250815 02:08:42.680817 7852 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:42.691043 7852 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 488c3a0d671942e3bbead647c317bc0a: Generated new TSK 0
I20250815 02:08:42.691286 7852 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250815 02:08:42.695766 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.129:0
--local_ip_for_outbound_sockets=127.6.59.129
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:40299
--builtin_ntp_servers=127.6.59.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:42.774086 7865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:42.774252 7865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:42.774284 7865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:42.775650 7865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:42.775708 7865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.129
I20250815 02:08:42.777138 7865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:40299
--never_fsync=true
--heap_profile_path=/tmp/kudu.7865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.129
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:42.777355 7865 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:42.777570 7865 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:42.779971 7870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:42.780179 7865 server_base.cc:1047] running on GCE node
W20250815 02:08:42.779979 7873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:42.779979 7871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:42.780464 7865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:42.780686 7865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:42.781810 7865 hybrid_clock.cc:648] HybridClock initialized: now 1755223722781794 us; error 33 us; skew 500 ppm
I20250815 02:08:42.783000 7865 webserver.cc:480] Webserver started at http://127.6.59.129:45839/ using document root <none> and password file <none>
I20250815 02:08:42.783205 7865 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:42.783244 7865 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:42.783346 7865 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:42.784188 7865 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "fb6336788026430dabfc7c0270d1e145"
format_stamp: "Formatted at 2025-08-15 02:08:42 on dist-test-slave-wq2q"
I20250815 02:08:42.784476 7865 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "fb6336788026430dabfc7c0270d1e145"
format_stamp: "Formatted at 2025-08-15 02:08:42 on dist-test-slave-wq2q"
I20250815 02:08:42.785620 7865 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250815 02:08:42.786372 7879 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:42.786525 7865 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:42.786579 7865 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "fb6336788026430dabfc7c0270d1e145"
format_stamp: "Formatted at 2025-08-15 02:08:42 on dist-test-slave-wq2q"
I20250815 02:08:42.786648 7865 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:42.798791 7865 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:42.799036 7865 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:42.799146 7865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:42.799343 7865 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:42.799650 7865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:42.799683 7865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:42.799721 7865 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:42.799752 7865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:42.805140 7865 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.129:46709
I20250815 02:08:42.805205 7992 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.129:46709 every 8 connection(s)
I20250815 02:08:42.805469 7865 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250815 02:08:42.809765 7993 heartbeater.cc:344] Connected to a master server at 127.6.59.190:40299
I20250815 02:08:42.809873 7993 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:42.810111 7993 heartbeater.cc:507] Master 127.6.59.190:40299 requested a full tablet report, sending...
I20250815 02:08:42.810107 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 7865
I20250815 02:08:42.810171 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250815 02:08:42.810580 7805 ts_manager.cc:194] Registered new tserver with Master: fb6336788026430dabfc7c0270d1e145 (127.6.59.129:46709)
I20250815 02:08:42.811285 7805 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.129:51369
I20250815 02:08:42.811331 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.130:0
--local_ip_for_outbound_sockets=127.6.59.130
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:40299
--builtin_ntp_servers=127.6.59.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:42.888183 7996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:42.888353 7996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:42.888372 7996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:42.889744 7996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:42.889797 7996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.130
I20250815 02:08:42.891299 7996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:40299
--never_fsync=true
--heap_profile_path=/tmp/kudu.7996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.130
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:42.891511 7996 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:42.891721 7996 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:42.894264 8001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:42.894271 8002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:42.894482 7996 server_base.cc:1047] running on GCE node
W20250815 02:08:42.894279 8004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:42.894680 7996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:42.894891 7996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:42.896023 7996 hybrid_clock.cc:648] HybridClock initialized: now 1755223722896013 us; error 32 us; skew 500 ppm
I20250815 02:08:42.897068 7996 webserver.cc:480] Webserver started at http://127.6.59.130:45899/ using document root <none> and password file <none>
I20250815 02:08:42.897251 7996 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:42.897293 7996 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:42.897498 7996 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:42.898484 7996 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "1d2b10448df34b2db68ad5f65412b4f3"
format_stamp: "Formatted at 2025-08-15 02:08:42 on dist-test-slave-wq2q"
I20250815 02:08:42.898833 7996 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "1d2b10448df34b2db68ad5f65412b4f3"
format_stamp: "Formatted at 2025-08-15 02:08:42 on dist-test-slave-wq2q"
I20250815 02:08:42.900069 7996 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:42.900782 8010 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:42.900933 7996 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:42.901005 7996 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "1d2b10448df34b2db68ad5f65412b4f3"
format_stamp: "Formatted at 2025-08-15 02:08:42 on dist-test-slave-wq2q"
I20250815 02:08:42.901065 7996 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:42.914232 7996 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:42.914494 7996 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:42.914606 7996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:42.914844 7996 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:42.915158 7996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:42.915190 7996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:42.915228 7996 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:42.915259 7996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:42.920676 7996 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.130:33713
I20250815 02:08:42.920747 8123 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.130:33713 every 8 connection(s)
I20250815 02:08:42.921030 7996 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250815 02:08:42.925393 8124 heartbeater.cc:344] Connected to a master server at 127.6.59.190:40299
I20250815 02:08:42.925462 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 7996
I20250815 02:08:42.925493 8124 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:42.925554 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250815 02:08:42.925722 8124 heartbeater.cc:507] Master 127.6.59.190:40299 requested a full tablet report, sending...
I20250815 02:08:42.926155 7804 ts_manager.cc:194] Registered new tserver with Master: 1d2b10448df34b2db68ad5f65412b4f3 (127.6.59.130:33713)
I20250815 02:08:42.926658 7804 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.130:40217
I20250815 02:08:42.926720 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.131:0
--local_ip_for_outbound_sockets=127.6.59.131
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:40299
--builtin_ntp_servers=127.6.59.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:43.004674 8127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:43.004877 8127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:43.004899 8127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:43.006279 8127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:43.006328 8127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.131
I20250815 02:08:43.007807 8127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:40299
--never_fsync=true
--heap_profile_path=/tmp/kudu.8127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.131
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:43.007979 8127 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:43.008160 8127 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:43.010730 8133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:43.010718 8135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:43.010720 8132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:43.010748 8127 server_base.cc:1047] running on GCE node
I20250815 02:08:43.011027 8127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:43.011197 8127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:43.012322 8127 hybrid_clock.cc:648] HybridClock initialized: now 1755223723012311 us; error 31 us; skew 500 ppm
I20250815 02:08:43.013414 8127 webserver.cc:480] Webserver started at http://127.6.59.131:33035/ using document root <none> and password file <none>
I20250815 02:08:43.013598 8127 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:43.013643 8127 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:43.013756 8127 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:43.014647 8127 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "ebd9c797df6947ca8d773accea7a796e"
format_stamp: "Formatted at 2025-08-15 02:08:43 on dist-test-slave-wq2q"
I20250815 02:08:43.014957 8127 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "ebd9c797df6947ca8d773accea7a796e"
format_stamp: "Formatted at 2025-08-15 02:08:43 on dist-test-slave-wq2q"
I20250815 02:08:43.016045 8127 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250815 02:08:43.016691 8141 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:43.016839 8127 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:43.016914 8127 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "ebd9c797df6947ca8d773accea7a796e"
format_stamp: "Formatted at 2025-08-15 02:08:43 on dist-test-slave-wq2q"
I20250815 02:08:43.016973 8127 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:43.046808 8127 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:43.047099 8127 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:43.047219 8127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:43.047425 8127 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:43.047791 8127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:43.047880 8127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:43.047966 8127 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:43.047996 8127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:43.053428 8127 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.131:44369
I20250815 02:08:43.053495 8254 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.131:44369 every 8 connection(s)
I20250815 02:08:43.053772 8127 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250815 02:08:43.057875 8255 heartbeater.cc:344] Connected to a master server at 127.6.59.190:40299
I20250815 02:08:43.057961 8255 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:43.058156 8255 heartbeater.cc:507] Master 127.6.59.190:40299 requested a full tablet report, sending...
I20250815 02:08:43.058482 7804 ts_manager.cc:194] Registered new tserver with Master: ebd9c797df6947ca8d773accea7a796e (127.6.59.131:44369)
I20250815 02:08:43.058846 7804 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.131:49625
I20250815 02:08:43.061051 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 8127
I20250815 02:08:43.061136 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250815 02:08:43.062077 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:43.067333 7804 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35416:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20250815 02:08:43.068557 7804 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250815 02:08:43.074883 8058 tablet_service.cc:1468] Processing CreateTablet for tablet 784e52d96d6245769684f07c1c035723 (DEFAULT_TABLE table=TestTable [id=bd34333447104fe4af6dea681d221a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:43.075038 7927 tablet_service.cc:1468] Processing CreateTablet for tablet 784e52d96d6245769684f07c1c035723 (DEFAULT_TABLE table=TestTable [id=bd34333447104fe4af6dea681d221a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:43.075227 8058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784e52d96d6245769684f07c1c035723. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:43.075325 7927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784e52d96d6245769684f07c1c035723. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:43.077247 8270 tablet_bootstrap.cc:492] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3: Bootstrap starting.
I20250815 02:08:43.077646 8189 tablet_service.cc:1468] Processing CreateTablet for tablet 784e52d96d6245769684f07c1c035723 (DEFAULT_TABLE table=TestTable [id=bd34333447104fe4af6dea681d221a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:43.077881 8189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784e52d96d6245769684f07c1c035723. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:43.077931 8270 tablet_bootstrap.cc:654] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:43.078195 8270 log.cc:826] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:43.078490 8269 tablet_bootstrap.cc:492] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145: Bootstrap starting.
I20250815 02:08:43.078830 8270 tablet_bootstrap.cc:492] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3: No bootstrap required, opened a new log
I20250815 02:08:43.078907 8270 ts_tablet_manager.cc:1397] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:43.079046 8269 tablet_bootstrap.cc:654] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:43.079293 8269 log.cc:826] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:43.079813 8269 tablet_bootstrap.cc:492] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145: No bootstrap required, opened a new log
I20250815 02:08:43.079866 8269 ts_tablet_manager.cc:1397] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:43.080456 8273 tablet_bootstrap.cc:492] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e: Bootstrap starting.
I20250815 02:08:43.080521 8270 raft_consensus.cc:357] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } }
I20250815 02:08:43.080657 8270 raft_consensus.cc:383] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:43.080709 8270 raft_consensus.cc:738] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2b10448df34b2db68ad5f65412b4f3, State: Initialized, Role: FOLLOWER
I20250815 02:08:43.080808 8270 consensus_queue.cc:260] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } }
I20250815 02:08:43.081032 8270 ts_tablet_manager.cc:1428] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250815 02:08:43.081034 8273 tablet_bootstrap.cc:654] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:43.081149 8124 heartbeater.cc:499] Master 127.6.59.190:40299 was elected leader, sending a full tablet report...
I20250815 02:08:43.081280 8273 log.cc:826] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:43.081456 8269 raft_consensus.cc:357] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } }
I20250815 02:08:43.081586 8269 raft_consensus.cc:383] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:43.081610 8269 raft_consensus.cc:738] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6336788026430dabfc7c0270d1e145, State: Initialized, Role: FOLLOWER
I20250815 02:08:43.081730 8269 consensus_queue.cc:260] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } }
I20250815 02:08:43.081944 8269 ts_tablet_manager.cc:1428] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250815 02:08:43.082054 7993 heartbeater.cc:499] Master 127.6.59.190:40299 was elected leader, sending a full tablet report...
I20250815 02:08:43.082437 8273 tablet_bootstrap.cc:492] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e: No bootstrap required, opened a new log
I20250815 02:08:43.082505 8273 ts_tablet_manager.cc:1397] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:43.083451 8273 raft_consensus.cc:357] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } }
I20250815 02:08:43.083581 8273 raft_consensus.cc:383] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:43.083624 8273 raft_consensus.cc:738] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebd9c797df6947ca8d773accea7a796e, State: Initialized, Role: FOLLOWER
I20250815 02:08:43.083709 8273 consensus_queue.cc:260] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } }
I20250815 02:08:43.083959 8273 ts_tablet_manager.cc:1428] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20250815 02:08:43.083973 8255 heartbeater.cc:499] Master 127.6.59.190:40299 was elected leader, sending a full tablet report...
I20250815 02:08:43.169114 8276 raft_consensus.cc:491] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250815 02:08:43.169241 8276 raft_consensus.cc:513] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } }
I20250815 02:08:43.169595 8276 leader_election.cc:290] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebd9c797df6947ca8d773accea7a796e (127.6.59.131:44369), 1d2b10448df34b2db68ad5f65412b4f3 (127.6.59.130:33713)
W20250815 02:08:43.171510 8125 tablet.cc:2378] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250815 02:08:43.172489 8209 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "784e52d96d6245769684f07c1c035723" candidate_uuid: "fb6336788026430dabfc7c0270d1e145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebd9c797df6947ca8d773accea7a796e" is_pre_election: true
I20250815 02:08:43.172663 8209 raft_consensus.cc:2466] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb6336788026430dabfc7c0270d1e145 in term 0.
I20250815 02:08:43.172899 7882 leader_election.cc:304] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ebd9c797df6947ca8d773accea7a796e, fb6336788026430dabfc7c0270d1e145; no voters:
I20250815 02:08:43.172901 8078 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "784e52d96d6245769684f07c1c035723" candidate_uuid: "fb6336788026430dabfc7c0270d1e145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d2b10448df34b2db68ad5f65412b4f3" is_pre_election: true
I20250815 02:08:43.173028 8078 raft_consensus.cc:2466] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb6336788026430dabfc7c0270d1e145 in term 0.
I20250815 02:08:43.173050 8276 raft_consensus.cc:2802] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250815 02:08:43.173127 8276 raft_consensus.cc:491] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250815 02:08:43.173180 8276 raft_consensus.cc:3058] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:43.173817 8276 raft_consensus.cc:513] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } }
I20250815 02:08:43.173941 8276 leader_election.cc:290] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [CANDIDATE]: Term 1 election: Requested vote from peers ebd9c797df6947ca8d773accea7a796e (127.6.59.131:44369), 1d2b10448df34b2db68ad5f65412b4f3 (127.6.59.130:33713)
I20250815 02:08:43.174137 8209 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "784e52d96d6245769684f07c1c035723" candidate_uuid: "fb6336788026430dabfc7c0270d1e145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebd9c797df6947ca8d773accea7a796e"
I20250815 02:08:43.174180 8078 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "784e52d96d6245769684f07c1c035723" candidate_uuid: "fb6336788026430dabfc7c0270d1e145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d2b10448df34b2db68ad5f65412b4f3"
I20250815 02:08:43.174230 8209 raft_consensus.cc:3058] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:43.174243 8078 raft_consensus.cc:3058] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:43.175010 8078 raft_consensus.cc:2466] T 784e52d96d6245769684f07c1c035723 P 1d2b10448df34b2db68ad5f65412b4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb6336788026430dabfc7c0270d1e145 in term 1.
I20250815 02:08:43.175010 8209 raft_consensus.cc:2466] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb6336788026430dabfc7c0270d1e145 in term 1.
I20250815 02:08:43.175218 7880 leader_election.cc:304] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d2b10448df34b2db68ad5f65412b4f3, fb6336788026430dabfc7c0270d1e145; no voters:
I20250815 02:08:43.175316 8276 raft_consensus.cc:2802] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:43.175442 8276 raft_consensus.cc:695] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [term 1 LEADER]: Becoming Leader. State: Replica: fb6336788026430dabfc7c0270d1e145, State: Running, Role: LEADER
I20250815 02:08:43.175529 8276 consensus_queue.cc:237] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } }
I20250815 02:08:43.176245 7805 catalog_manager.cc:5582] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb6336788026430dabfc7c0270d1e145 (127.6.59.129). New cstate: current_term: 1 leader_uuid: "fb6336788026430dabfc7c0270d1e145" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } health_report { overall_health: HEALTHY } } }
I20250815 02:08:43.188851 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:43.189308 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver fb6336788026430dabfc7c0270d1e145 to finish bootstrapping
I20250815 02:08:43.192746 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 1d2b10448df34b2db68ad5f65412b4f3 to finish bootstrapping
I20250815 02:08:43.195962 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver ebd9c797df6947ca8d773accea7a796e to finish bootstrapping
I20250815 02:08:43.198958 7805 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35416:
name: "TestAnotherTable"
schema {
columns {
name: "foo"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "bar"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
comment: "comment for bar"
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "foo"
}
}
}
W20250815 02:08:43.199112 7805 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestAnotherTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250815 02:08:43.200431 7927 tablet_service.cc:1468] Processing CreateTablet for tablet a2aad09433364fa79056f54338f3536e (DEFAULT_TABLE table=TestAnotherTable [id=786670765209493d9cab79c8bb5cfef8]), partition=RANGE (foo) PARTITION UNBOUNDED
I20250815 02:08:43.200599 7927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2aad09433364fa79056f54338f3536e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:43.200608 8189 tablet_service.cc:1468] Processing CreateTablet for tablet a2aad09433364fa79056f54338f3536e (DEFAULT_TABLE table=TestAnotherTable [id=786670765209493d9cab79c8bb5cfef8]), partition=RANGE (foo) PARTITION UNBOUNDED
I20250815 02:08:43.200726 8189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2aad09433364fa79056f54338f3536e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:43.201874 8269 tablet_bootstrap.cc:492] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145: Bootstrap starting.
I20250815 02:08:43.202075 8058 tablet_service.cc:1468] Processing CreateTablet for tablet a2aad09433364fa79056f54338f3536e (DEFAULT_TABLE table=TestAnotherTable [id=786670765209493d9cab79c8bb5cfef8]), partition=RANGE (foo) PARTITION UNBOUNDED
I20250815 02:08:43.202194 8058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2aad09433364fa79056f54338f3536e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:43.202358 8269 tablet_bootstrap.cc:654] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:43.202687 8273 tablet_bootstrap.cc:492] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e: Bootstrap starting.
I20250815 02:08:43.203017 8269 tablet_bootstrap.cc:492] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145: No bootstrap required, opened a new log
I20250815 02:08:43.203075 8269 ts_tablet_manager.cc:1397] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:43.203119 8273 tablet_bootstrap.cc:654] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:43.203241 8270 tablet_bootstrap.cc:492] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3: Bootstrap starting.
I20250815 02:08:43.203271 8269 raft_consensus.cc:357] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } }
I20250815 02:08:43.203354 8269 raft_consensus.cc:383] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:43.203382 8269 raft_consensus.cc:738] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6336788026430dabfc7c0270d1e145, State: Initialized, Role: FOLLOWER
I20250815 02:08:43.203437 8269 consensus_queue.cc:260] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } }
I20250815 02:08:43.203528 8269 ts_tablet_manager.cc:1428] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:43.203747 8270 tablet_bootstrap.cc:654] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:43.203807 8273 tablet_bootstrap.cc:492] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e: No bootstrap required, opened a new log
I20250815 02:08:43.203851 8273 ts_tablet_manager.cc:1397] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250815 02:08:43.204034 8273 raft_consensus.cc:357] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } }
I20250815 02:08:43.204088 8273 raft_consensus.cc:383] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:43.204102 8273 raft_consensus.cc:738] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebd9c797df6947ca8d773accea7a796e, State: Initialized, Role: FOLLOWER
I20250815 02:08:43.204139 8273 consensus_queue.cc:260] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } }
I20250815 02:08:43.204241 8273 ts_tablet_manager.cc:1428] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:43.204716 8270 tablet_bootstrap.cc:492] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3: No bootstrap required, opened a new log
I20250815 02:08:43.204766 8270 ts_tablet_manager.cc:1397] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:43.204936 8270 raft_consensus.cc:357] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } }
I20250815 02:08:43.205000 8270 raft_consensus.cc:383] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:43.205015 8270 raft_consensus.cc:738] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2b10448df34b2db68ad5f65412b4f3, State: Initialized, Role: FOLLOWER
I20250815 02:08:43.205081 8270 consensus_queue.cc:260] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } }
I20250815 02:08:43.205200 8270 ts_tablet_manager.cc:1428] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:43.234956 8274 raft_consensus.cc:491] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250815 02:08:43.235062 8274 raft_consensus.cc:513] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } }
I20250815 02:08:43.235427 8274 leader_election.cc:290] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6336788026430dabfc7c0270d1e145 (127.6.59.129:46709), ebd9c797df6947ca8d773accea7a796e (127.6.59.131:44369)
I20250815 02:08:43.238142 8209 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a2aad09433364fa79056f54338f3536e" candidate_uuid: "1d2b10448df34b2db68ad5f65412b4f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebd9c797df6947ca8d773accea7a796e" is_pre_election: true
I20250815 02:08:43.238238 8209 raft_consensus.cc:2466] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d2b10448df34b2db68ad5f65412b4f3 in term 0.
I20250815 02:08:43.238226 7947 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a2aad09433364fa79056f54338f3536e" candidate_uuid: "1d2b10448df34b2db68ad5f65412b4f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6336788026430dabfc7c0270d1e145" is_pre_election: true
I20250815 02:08:43.238350 7947 raft_consensus.cc:2466] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d2b10448df34b2db68ad5f65412b4f3 in term 0.
I20250815 02:08:43.238418 8013 leader_election.cc:304] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d2b10448df34b2db68ad5f65412b4f3, ebd9c797df6947ca8d773accea7a796e; no voters:
I20250815 02:08:43.238540 8274 raft_consensus.cc:2802] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250815 02:08:43.238583 8274 raft_consensus.cc:491] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250815 02:08:43.238603 8274 raft_consensus.cc:3058] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:43.239226 8274 raft_consensus.cc:513] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } }
I20250815 02:08:43.239351 8274 leader_election.cc:290] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [CANDIDATE]: Term 1 election: Requested vote from peers fb6336788026430dabfc7c0270d1e145 (127.6.59.129:46709), ebd9c797df6947ca8d773accea7a796e (127.6.59.131:44369)
I20250815 02:08:43.239480 8209 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a2aad09433364fa79056f54338f3536e" candidate_uuid: "1d2b10448df34b2db68ad5f65412b4f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebd9c797df6947ca8d773accea7a796e"
I20250815 02:08:43.239485 7947 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a2aad09433364fa79056f54338f3536e" candidate_uuid: "1d2b10448df34b2db68ad5f65412b4f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6336788026430dabfc7c0270d1e145"
I20250815 02:08:43.239538 8209 raft_consensus.cc:3058] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:43.239557 7947 raft_consensus.cc:3058] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:43.240067 8209 raft_consensus.cc:2466] T a2aad09433364fa79056f54338f3536e P ebd9c797df6947ca8d773accea7a796e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d2b10448df34b2db68ad5f65412b4f3 in term 1.
I20250815 02:08:43.240146 7947 raft_consensus.cc:2466] T a2aad09433364fa79056f54338f3536e P fb6336788026430dabfc7c0270d1e145 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d2b10448df34b2db68ad5f65412b4f3 in term 1.
I20250815 02:08:43.240252 8013 leader_election.cc:304] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d2b10448df34b2db68ad5f65412b4f3, ebd9c797df6947ca8d773accea7a796e; no voters:
I20250815 02:08:43.240338 8274 raft_consensus.cc:2802] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:43.240499 8274 raft_consensus.cc:695] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [term 1 LEADER]: Becoming Leader. State: Replica: 1d2b10448df34b2db68ad5f65412b4f3, State: Running, Role: LEADER
I20250815 02:08:43.240600 8274 consensus_queue.cc:237] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } } peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } }
I20250815 02:08:43.241236 7805 catalog_manager.cc:5582] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d2b10448df34b2db68ad5f65412b4f3 (127.6.59.130). New cstate: current_term: 1 leader_uuid: "1d2b10448df34b2db68ad5f65412b4f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 } health_report { overall_health: UNKNOWN } } }
W20250815 02:08:43.304319 8256 tablet.cc:2378] T 784e52d96d6245769684f07c1c035723 P ebd9c797df6947ca8d773accea7a796e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250815 02:08:43.306092 7994 tablet.cc:2378] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250815 02:08:43.322306 8288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:43.322477 8288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:43.323789 8288 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250815 02:08:43.465602 8298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:43.465773 8298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:43.467142 8298 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250815 02:08:43.608719 8308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:43.608887 8308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:43.610297 8308 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20250815 02:08:43.644887 8274 consensus_queue.cc:1035] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:43.646917 8286 consensus_queue.cc:1035] T a2aad09433364fa79056f54338f3536e P 1d2b10448df34b2db68ad5f65412b4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6336788026430dabfc7c0270d1e145" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 46709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:43.681303 8276 consensus_queue.cc:1035] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d2b10448df34b2db68ad5f65412b4f3" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:43.682911 8325 consensus_queue.cc:1035] T 784e52d96d6245769684f07c1c035723 P fb6336788026430dabfc7c0270d1e145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebd9c797df6947ca8d773accea7a796e" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 44369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250815 02:08:43.753363 8323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:43.753535 8323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:43.754932 8323 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20250815 02:08:43.820729 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 7865
I20250815 02:08:43.825552 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 7996
I20250815 02:08:43.829972 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 8127
I20250815 02:08:43.834559 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 7774
2025-08-15T02:08:43Z chronyd exiting
[ OK ] AdminCliTest.TestDescribeTableColumnFlags (1335 ms)
[ RUN ] AdminCliTest.TestAuthzResetCacheNotAuthorized
I20250815 02:08:43.854566 6382 test_util.cc:276] Using random seed: -561001758
I20250815 02:08:43.855105 6382 ts_itest-base.cc:115] Starting cluster with:
I20250815 02:08:43.855147 6382 ts_itest-base.cc:116] --------------
I20250815 02:08:43.855165 6382 ts_itest-base.cc:117] 3 tablet servers
I20250815 02:08:43.855181 6382 ts_itest-base.cc:118] 3 replicas per TS
I20250815 02:08:43.855206 6382 ts_itest-base.cc:119] --------------
2025-08-15T02:08:43Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-15T02:08:43Z Disabled control of system clock
I20250815 02:08:43.861598 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:35059
--webserver_interface=127.6.59.190
--webserver_port=0
--builtin_ntp_servers=127.6.59.148:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:35059
--superuser_acl=no-such-user with env {}
W20250815 02:08:43.936477 8343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:43.936648 8343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:43.936676 8343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:43.938050 8343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:43.938089 8343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:43.938109 8343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:43.938129 8343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:43.939584 8343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:35059
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:35059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.8343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:43.939832 8343 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:43.940054 8343 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:43.942378 8349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:43.942386 8348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:43.942482 8343 server_base.cc:1047] running on GCE node
W20250815 02:08:43.942448 8351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:43.942817 8343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:43.943061 8343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:43.944191 8343 hybrid_clock.cc:648] HybridClock initialized: now 1755223723944183 us; error 28 us; skew 500 ppm
I20250815 02:08:43.945276 8343 webserver.cc:480] Webserver started at http://127.6.59.190:33565/ using document root <none> and password file <none>
I20250815 02:08:43.945473 8343 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:43.945516 8343 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:43.945626 8343 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:43.946504 8343 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "bad5f380b81141dab4d7066d139c9a3b"
format_stamp: "Formatted at 2025-08-15 02:08:43 on dist-test-slave-wq2q"
I20250815 02:08:43.946790 8343 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "bad5f380b81141dab4d7066d139c9a3b"
format_stamp: "Formatted at 2025-08-15 02:08:43 on dist-test-slave-wq2q"
I20250815 02:08:43.947870 8343 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:43.948513 8357 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:43.948693 8343 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:43.948765 8343 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
uuid: "bad5f380b81141dab4d7066d139c9a3b"
format_stamp: "Formatted at 2025-08-15 02:08:43 on dist-test-slave-wq2q"
I20250815 02:08:43.948823 8343 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:43.969810 8343 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:43.970141 8343 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:43.970264 8343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:43.973819 8343 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:35059
I20250815 02:08:43.973896 8409 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:35059 every 8 connection(s)
I20250815 02:08:43.974228 8343 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250815 02:08:43.974795 8410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:43.975170 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 8343
I20250815 02:08:43.975265 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250815 02:08:43.977528 8410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b: Bootstrap starting.
I20250815 02:08:43.978165 8410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:43.978406 8410 log.cc:826] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:43.978992 8410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b: No bootstrap required, opened a new log
I20250815 02:08:43.980151 8410 raft_consensus.cc:357] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad5f380b81141dab4d7066d139c9a3b" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35059 } }
I20250815 02:08:43.980266 8410 raft_consensus.cc:383] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:43.980285 8410 raft_consensus.cc:738] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad5f380b81141dab4d7066d139c9a3b, State: Initialized, Role: FOLLOWER
I20250815 02:08:43.980353 8410 consensus_queue.cc:260] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad5f380b81141dab4d7066d139c9a3b" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35059 } }
I20250815 02:08:43.980451 8410 raft_consensus.cc:397] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:43.980495 8410 raft_consensus.cc:491] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:43.980528 8410 raft_consensus.cc:3058] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:43.980993 8410 raft_consensus.cc:513] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad5f380b81141dab4d7066d139c9a3b" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35059 } }
I20250815 02:08:43.981086 8410 leader_election.cc:304] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bad5f380b81141dab4d7066d139c9a3b; no voters:
I20250815 02:08:43.981230 8410 leader_election.cc:290] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:43.981272 8415 raft_consensus.cc:2802] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:43.981407 8415 raft_consensus.cc:695] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [term 1 LEADER]: Becoming Leader. State: Replica: bad5f380b81141dab4d7066d139c9a3b, State: Running, Role: LEADER
I20250815 02:08:43.981433 8410 sys_catalog.cc:564] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:43.981523 8415 consensus_queue.cc:237] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad5f380b81141dab4d7066d139c9a3b" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35059 } }
I20250815 02:08:43.982013 8416 sys_catalog.cc:455] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bad5f380b81141dab4d7066d139c9a3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad5f380b81141dab4d7066d139c9a3b" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35059 } } }
I20250815 02:08:43.982030 8417 sys_catalog.cc:455] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bad5f380b81141dab4d7066d139c9a3b. Latest consensus state: current_term: 1 leader_uuid: "bad5f380b81141dab4d7066d139c9a3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad5f380b81141dab4d7066d139c9a3b" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 35059 } } }
I20250815 02:08:43.982224 8416 sys_catalog.cc:458] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:43.982237 8417 sys_catalog.cc:458] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:43.982620 8424 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:43.983047 8424 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:43.984338 8424 catalog_manager.cc:1349] Generated new cluster ID: 0caa2e0500a447ff8e6d746ca83b6b82
I20250815 02:08:43.984385 8424 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:43.992703 8424 catalog_manager.cc:1372] Generated new certificate authority record
I20250815 02:08:43.993376 8424 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:44.000768 8424 catalog_manager.cc:5955] T 00000000000000000000000000000000 P bad5f380b81141dab4d7066d139c9a3b: Generated new TSK 0
I20250815 02:08:44.000968 8424 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250815 02:08:44.007328 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.129:0
--local_ip_for_outbound_sockets=127.6.59.129
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35059
--builtin_ntp_servers=127.6.59.148:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:44.082825 8434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:44.082993 8434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:44.083012 8434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:44.084378 8434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:44.084431 8434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.129
I20250815 02:08:44.085763 8434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35059
--never_fsync=true
--heap_profile_path=/tmp/kudu.8434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.129
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:44.085991 8434 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:44.086222 8434 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:44.088680 8439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:44.088714 8440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:44.088801 8434 server_base.cc:1047] running on GCE node
W20250815 02:08:44.088690 8442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:44.089064 8434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:44.089262 8434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:44.090399 8434 hybrid_clock.cc:648] HybridClock initialized: now 1755223724090379 us; error 35 us; skew 500 ppm
I20250815 02:08:44.091538 8434 webserver.cc:480] Webserver started at http://127.6.59.129:37541/ using document root <none> and password file <none>
I20250815 02:08:44.091730 8434 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:44.091774 8434 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:44.091893 8434 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:44.092767 8434 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "49daec66ba2f40bd972b09f95b86f1bc"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.093050 8434 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "49daec66ba2f40bd972b09f95b86f1bc"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.094178 8434 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:44.094907 8448 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.095082 8434 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:44.095153 8434 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "49daec66ba2f40bd972b09f95b86f1bc"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.095211 8434 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:44.102262 8434 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:44.102485 8434 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:44.102589 8434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:44.102806 8434 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:44.103086 8434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:44.103119 8434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.103152 8434 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:44.103181 8434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.110044 8434 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.129:45219
I20250815 02:08:44.110415 8434 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250815 02:08:44.111822 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 8434
I20250815 02:08:44.111900 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250815 02:08:44.112210 8561 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.129:45219 every 8 connection(s)
I20250815 02:08:44.114435 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.130:0
--local_ip_for_outbound_sockets=127.6.59.130
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35059
--builtin_ntp_servers=127.6.59.148:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250815 02:08:44.118180 8562 heartbeater.cc:344] Connected to a master server at 127.6.59.190:35059
I20250815 02:08:44.118273 8562 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:44.118467 8562 heartbeater.cc:507] Master 127.6.59.190:35059 requested a full tablet report, sending...
I20250815 02:08:44.118932 8374 ts_manager.cc:194] Registered new tserver with Master: 49daec66ba2f40bd972b09f95b86f1bc (127.6.59.129:45219)
I20250815 02:08:44.119820 8374 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.129:35585
W20250815 02:08:44.201576 8565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:44.201759 8565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:44.201777 8565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:44.203225 8565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:44.203279 8565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.130
I20250815 02:08:44.204648 8565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35059
--never_fsync=true
--heap_profile_path=/tmp/kudu.8565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.130
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:44.204825 8565 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:44.205008 8565 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:44.207340 8573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:44.207376 8570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:44.207430 8565 server_base.cc:1047] running on GCE node
W20250815 02:08:44.207386 8571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:44.207667 8565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:44.207863 8565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:44.208989 8565 hybrid_clock.cc:648] HybridClock initialized: now 1755223724208980 us; error 30 us; skew 500 ppm
I20250815 02:08:44.210078 8565 webserver.cc:480] Webserver started at http://127.6.59.130:41047/ using document root <none> and password file <none>
I20250815 02:08:44.210287 8565 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:44.210333 8565 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:44.210448 8565 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:44.211256 8565 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "e24fccf4d8ac4765a4850b48fa5830dd"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.211522 8565 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "e24fccf4d8ac4765a4850b48fa5830dd"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.212607 8565 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:44.213244 8579 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.213399 8565 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:44.213467 8565 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "e24fccf4d8ac4765a4850b48fa5830dd"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.213523 8565 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:44.233495 8565 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:44.233752 8565 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:44.233865 8565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:44.234126 8565 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:44.234449 8565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:44.234483 8565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.234520 8565 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:44.234546 8565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.239879 8565 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.130:41121
I20250815 02:08:44.239938 8692 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.130:41121 every 8 connection(s)
I20250815 02:08:44.240221 8565 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250815 02:08:44.244690 8693 heartbeater.cc:344] Connected to a master server at 127.6.59.190:35059
I20250815 02:08:44.244786 8693 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:44.244975 8693 heartbeater.cc:507] Master 127.6.59.190:35059 requested a full tablet report, sending...
I20250815 02:08:44.245358 8374 ts_manager.cc:194] Registered new tserver with Master: e24fccf4d8ac4765a4850b48fa5830dd (127.6.59.130:41121)
I20250815 02:08:44.245736 8374 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.130:39879
I20250815 02:08:44.249727 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 8565
I20250815 02:08:44.249814 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250815 02:08:44.250762 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.131:0
--local_ip_for_outbound_sockets=127.6.59.131
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35059
--builtin_ntp_servers=127.6.59.148:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:44.328181 8696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:44.328382 8696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:44.328418 8696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:44.329895 8696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:44.329967 8696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.131
I20250815 02:08:44.331538 8696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:35059
--never_fsync=true
--heap_profile_path=/tmp/kudu.8696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.131
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:44.331813 8696 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:44.332052 8696 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:44.334688 8701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:44.334688 8704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:44.334683 8702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:44.335228 8696 server_base.cc:1047] running on GCE node
I20250815 02:08:44.335438 8696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:44.335661 8696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:44.336827 8696 hybrid_clock.cc:648] HybridClock initialized: now 1755223724336769 us; error 83 us; skew 500 ppm
I20250815 02:08:44.338166 8696 webserver.cc:480] Webserver started at http://127.6.59.131:40991/ using document root <none> and password file <none>
I20250815 02:08:44.338392 8696 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:44.338438 8696 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:44.338552 8696 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:44.339449 8696 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "097f7242136d4e71a06a53495931555b"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.339761 8696 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "097f7242136d4e71a06a53495931555b"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.341007 8696 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20250815 02:08:44.341863 8710 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.342082 8696 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:44.342172 8696 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "097f7242136d4e71a06a53495931555b"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.342240 8696 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:44.354876 8696 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:44.355154 8696 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:44.355271 8696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:44.355496 8696 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:44.355868 8696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:44.355907 8696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.355955 8696 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:44.355981 8696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.362200 8696 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.131:45053
I20250815 02:08:44.362277 8823 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.131:45053 every 8 connection(s)
I20250815 02:08:44.362604 8696 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250815 02:08:44.364806 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 8696
I20250815 02:08:44.364907 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250815 02:08:44.367559 8824 heartbeater.cc:344] Connected to a master server at 127.6.59.190:35059
I20250815 02:08:44.367662 8824 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:44.367838 8824 heartbeater.cc:507] Master 127.6.59.190:35059 requested a full tablet report, sending...
I20250815 02:08:44.368147 8374 ts_manager.cc:194] Registered new tserver with Master: 097f7242136d4e71a06a53495931555b (127.6.59.131:45053)
I20250815 02:08:44.368522 8374 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.131:54223
I20250815 02:08:44.377210 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:44.382797 8374 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39912:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20250815 02:08:44.384155 8374 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250815 02:08:44.390508 8627 tablet_service.cc:1468] Processing CreateTablet for tablet d06176a782e143ee8b77c9015e2712c9 (DEFAULT_TABLE table=TestTable [id=d301714c61f04ecf8843bc6c18dd9946]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:44.390877 8627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06176a782e143ee8b77c9015e2712c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:44.390936 8758 tablet_service.cc:1468] Processing CreateTablet for tablet d06176a782e143ee8b77c9015e2712c9 (DEFAULT_TABLE table=TestTable [id=d301714c61f04ecf8843bc6c18dd9946]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:44.391042 8496 tablet_service.cc:1468] Processing CreateTablet for tablet d06176a782e143ee8b77c9015e2712c9 (DEFAULT_TABLE table=TestTable [id=d301714c61f04ecf8843bc6c18dd9946]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:44.391153 8758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06176a782e143ee8b77c9015e2712c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:44.391291 8496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06176a782e143ee8b77c9015e2712c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:44.393105 8838 tablet_bootstrap.cc:492] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b: Bootstrap starting.
I20250815 02:08:44.393661 8839 tablet_bootstrap.cc:492] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd: Bootstrap starting.
I20250815 02:08:44.393731 8838 tablet_bootstrap.cc:654] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:44.394269 8838 log.cc:826] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:44.394304 8840 tablet_bootstrap.cc:492] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc: Bootstrap starting.
I20250815 02:08:44.394347 8839 tablet_bootstrap.cc:654] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:44.394613 8839 log.cc:826] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:44.394961 8840 tablet_bootstrap.cc:654] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:44.395030 8838 tablet_bootstrap.cc:492] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b: No bootstrap required, opened a new log
I20250815 02:08:44.395102 8838 ts_tablet_manager.cc:1397] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:44.395244 8839 tablet_bootstrap.cc:492] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd: No bootstrap required, opened a new log
I20250815 02:08:44.395305 8839 ts_tablet_manager.cc:1397] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250815 02:08:44.395393 8840 log.cc:826] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:44.396111 8840 tablet_bootstrap.cc:492] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc: No bootstrap required, opened a new log
I20250815 02:08:44.396174 8840 ts_tablet_manager.cc:1397] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:44.396654 8838 raft_consensus.cc:357] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 41121 } } peers { permanent_uuid: "49daec66ba2f40bd972b09f95b86f1bc" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 45219 } } peers { permanent_uuid: "097f7242136d4e71a06a53495931555b" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 45053 } }
I20250815 02:08:44.396678 8839 raft_consensus.cc:357] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 41121 } } peers { permanent_uuid: "49daec66ba2f40bd972b09f95b86f1bc" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 45219 } } peers { permanent_uuid: "097f7242136d4e71a06a53495931555b" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 45053 } }
I20250815 02:08:44.396821 8839 raft_consensus.cc:383] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:44.396876 8839 raft_consensus.cc:738] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e24fccf4d8ac4765a4850b48fa5830dd, State: Initialized, Role: FOLLOWER
I20250815 02:08:44.396973 8838 raft_consensus.cc:383] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:44.396999 8839 consensus_queue.cc:260] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 41121 } } peers { permanent_uuid: "49daec66ba2f40bd972b09f95b86f1bc" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 45219 } } peers { permanent_uuid: "097f7242136d4e71a06a53495931555b" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 45053 } }
I20250815 02:08:44.397068 8838 raft_consensus.cc:738] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 097f7242136d4e71a06a53495931555b, State: Initialized, Role: FOLLOWER
I20250815 02:08:44.397176 8838 consensus_queue.cc:260] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 41121 } } peers { permanent_uuid: "49daec66ba2f40bd972b09f95b86f1bc" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 45219 } } peers { permanent_uuid: "097f7242136d4e71a06a53495931555b" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 45053 } }
I20250815 02:08:44.397359 8839 ts_tablet_manager.cc:1428] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250815 02:08:44.397398 8838 ts_tablet_manager.cc:1428] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250815 02:08:44.397444 8824 heartbeater.cc:499] Master 127.6.59.190:35059 was elected leader, sending a full tablet report...
I20250815 02:08:44.397516 8693 heartbeater.cc:499] Master 127.6.59.190:35059 was elected leader, sending a full tablet report...
I20250815 02:08:44.397588 8840 raft_consensus.cc:357] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 41121 } } peers { permanent_uuid: "49daec66ba2f40bd972b09f95b86f1bc" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 45219 } } peers { permanent_uuid: "097f7242136d4e71a06a53495931555b" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 45053 } }
I20250815 02:08:44.397761 8840 raft_consensus.cc:383] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:44.397832 8840 raft_consensus.cc:738] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49daec66ba2f40bd972b09f95b86f1bc, State: Initialized, Role: FOLLOWER
I20250815 02:08:44.397953 8840 consensus_queue.cc:260] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 41121 } } peers { permanent_uuid: "49daec66ba2f40bd972b09f95b86f1bc" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 45219 } } peers { permanent_uuid: "097f7242136d4e71a06a53495931555b" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 45053 } }
I20250815 02:08:44.398268 8840 ts_tablet_manager.cc:1428] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250815 02:08:44.398415 8562 heartbeater.cc:499] Master 127.6.59.190:35059 was elected leader, sending a full tablet report...
W20250815 02:08:44.490685 8694 tablet.cc:2378] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250815 02:08:44.516348 8844 raft_consensus.cc:491] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250815 02:08:44.516472 8844 raft_consensus.cc:513] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 41121 } } peers { permanent_uuid: "49daec66ba2f40bd972b09f95b86f1bc" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 45219 } } peers { permanent_uuid: "097f7242136d4e71a06a53495931555b" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 45053 } }
I20250815 02:08:44.516800 8844 leader_election.cc:290] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49daec66ba2f40bd972b09f95b86f1bc (127.6.59.129:45219), 097f7242136d4e71a06a53495931555b (127.6.59.131:45053)
I20250815 02:08:44.519878 8778 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d06176a782e143ee8b77c9015e2712c9" candidate_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "097f7242136d4e71a06a53495931555b" is_pre_election: true
I20250815 02:08:44.520061 8778 raft_consensus.cc:2466] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e24fccf4d8ac4765a4850b48fa5830dd in term 0.
I20250815 02:08:44.520159 8514 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d06176a782e143ee8b77c9015e2712c9" candidate_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49daec66ba2f40bd972b09f95b86f1bc" is_pre_election: true
I20250815 02:08:44.520287 8583 leader_election.cc:304] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 097f7242136d4e71a06a53495931555b, e24fccf4d8ac4765a4850b48fa5830dd; no voters:
I20250815 02:08:44.520323 8514 raft_consensus.cc:2466] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e24fccf4d8ac4765a4850b48fa5830dd in term 0.
I20250815 02:08:44.520424 8844 raft_consensus.cc:2802] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250815 02:08:44.520476 8844 raft_consensus.cc:491] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250815 02:08:44.520496 8844 raft_consensus.cc:3058] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:44.521126 8844 raft_consensus.cc:513] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 41121 } } peers { permanent_uuid: "49daec66ba2f40bd972b09f95b86f1bc" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 45219 } } peers { permanent_uuid: "097f7242136d4e71a06a53495931555b" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 45053 } }
I20250815 02:08:44.521246 8844 leader_election.cc:290] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [CANDIDATE]: Term 1 election: Requested vote from peers 49daec66ba2f40bd972b09f95b86f1bc (127.6.59.129:45219), 097f7242136d4e71a06a53495931555b (127.6.59.131:45053)
I20250815 02:08:44.521387 8514 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d06176a782e143ee8b77c9015e2712c9" candidate_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49daec66ba2f40bd972b09f95b86f1bc"
I20250815 02:08:44.521390 8778 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "d06176a782e143ee8b77c9015e2712c9" candidate_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "097f7242136d4e71a06a53495931555b"
I20250815 02:08:44.521447 8778 raft_consensus.cc:3058] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:44.521450 8514 raft_consensus.cc:3058] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:44.522048 8778 raft_consensus.cc:2466] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e24fccf4d8ac4765a4850b48fa5830dd in term 1.
I20250815 02:08:44.522104 8514 raft_consensus.cc:2466] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e24fccf4d8ac4765a4850b48fa5830dd in term 1.
I20250815 02:08:44.522230 8583 leader_election.cc:304] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 097f7242136d4e71a06a53495931555b, e24fccf4d8ac4765a4850b48fa5830dd; no voters:
I20250815 02:08:44.522333 8844 raft_consensus.cc:2802] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:44.522529 8844 raft_consensus.cc:695] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [term 1 LEADER]: Becoming Leader. State: Replica: e24fccf4d8ac4765a4850b48fa5830dd, State: Running, Role: LEADER
I20250815 02:08:44.522634 8844 consensus_queue.cc:237] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 41121 } } peers { permanent_uuid: "49daec66ba2f40bd972b09f95b86f1bc" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 45219 } } peers { permanent_uuid: "097f7242136d4e71a06a53495931555b" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 45053 } }
I20250815 02:08:44.523532 8373 catalog_manager.cc:5582] T d06176a782e143ee8b77c9015e2712c9 P e24fccf4d8ac4765a4850b48fa5830dd reported cstate change: term changed from 0 to 1, leader changed from <none> to e24fccf4d8ac4765a4850b48fa5830dd (127.6.59.130). New cstate: current_term: 1 leader_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24fccf4d8ac4765a4850b48fa5830dd" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 41121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49daec66ba2f40bd972b09f95b86f1bc" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "097f7242136d4e71a06a53495931555b" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 45053 } health_report { overall_health: UNKNOWN } } }
I20250815 02:08:44.535550 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:44.536049 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 49daec66ba2f40bd972b09f95b86f1bc to finish bootstrapping
I20250815 02:08:44.539539 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e24fccf4d8ac4765a4850b48fa5830dd to finish bootstrapping
I20250815 02:08:44.542445 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 097f7242136d4e71a06a53495931555b to finish bootstrapping
W20250815 02:08:44.612964 8563 tablet.cc:2378] T d06176a782e143ee8b77c9015e2712c9 P 49daec66ba2f40bd972b09f95b86f1bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250815 02:08:44.613361 8825 tablet.cc:2378] T d06176a782e143ee8b77c9015e2712c9 P 097f7242136d4e71a06a53495931555b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250815 02:08:44.635777 8373 server_base.cc:1129] Unauthorized access attempt to method kudu.master.MasterService.RefreshAuthzCache from {username='slave'} at 127.0.0.1:39920
I20250815 02:08:44.691170 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 8434
I20250815 02:08:44.695803 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 8565
I20250815 02:08:44.700187 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 8696
I20250815 02:08:44.704300 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 8343
2025-08-15T02:08:44Z chronyd exiting
[ OK ] AdminCliTest.TestAuthzResetCacheNotAuthorized (868 ms)
[ RUN ] AdminCliTest.TestRebuildTables
I20250815 02:08:44.723336 6382 test_util.cc:276] Using random seed: -560132989
I20250815 02:08:44.723845 6382 ts_itest-base.cc:115] Starting cluster with:
I20250815 02:08:44.723884 6382 ts_itest-base.cc:116] --------------
I20250815 02:08:44.723896 6382 ts_itest-base.cc:117] 3 tablet servers
I20250815 02:08:44.723905 6382 ts_itest-base.cc:118] 3 replicas per TS
I20250815 02:08:44.723932 6382 ts_itest-base.cc:119] --------------
2025-08-15T02:08:44Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-15T02:08:44Z Disabled control of system clock
I20250815 02:08:44.730660 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:42923
--webserver_interface=127.6.59.190
--webserver_port=0
--builtin_ntp_servers=127.6.59.148:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:42923 with env {}
W20250815 02:08:44.805372 8873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:44.805532 8873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:44.805562 8873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:44.806914 8873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:44.806957 8873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:44.806979 8873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:44.806996 8873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:44.808388 8873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:42923
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:42923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.8873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:44.808636 8873 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:44.808856 8873 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:44.811160 8878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:44.811156 8879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:44.811244 8873 server_base.cc:1047] running on GCE node
W20250815 02:08:44.811198 8881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:44.811614 8873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:44.811813 8873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:44.812949 8873 hybrid_clock.cc:648] HybridClock initialized: now 1755223724812935 us; error 27 us; skew 500 ppm
I20250815 02:08:44.814105 8873 webserver.cc:480] Webserver started at http://127.6.59.190:45445/ using document root <none> and password file <none>
I20250815 02:08:44.814296 8873 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:44.814340 8873 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:44.814451 8873 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:44.815289 8873 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "e6537b4139ee49ebaaf570e251a9fb8a"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.815574 8873 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "e6537b4139ee49ebaaf570e251a9fb8a"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.816653 8873 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250815 02:08:44.817293 8887 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.817443 8873 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:44.817513 8873 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
uuid: "e6537b4139ee49ebaaf570e251a9fb8a"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.817571 8873 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:44.827755 8873 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:44.827987 8873 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:44.828091 8873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:44.831684 8873 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:42923
I20250815 02:08:44.831743 8939 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:42923 every 8 connection(s)
I20250815 02:08:44.832021 8873 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250815 02:08:44.832520 8940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:44.834103 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 8873
I20250815 02:08:44.834193 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250815 02:08:44.835079 8940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Bootstrap starting.
I20250815 02:08:44.835721 8940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:44.836019 8940 log.cc:826] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:44.836575 8940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: No bootstrap required, opened a new log
I20250815 02:08:44.837925 8940 raft_consensus.cc:357] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:44.838093 8940 raft_consensus.cc:383] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:44.838137 8940 raft_consensus.cc:738] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6537b4139ee49ebaaf570e251a9fb8a, State: Initialized, Role: FOLLOWER
I20250815 02:08:44.838232 8940 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:44.838284 8940 raft_consensus.cc:397] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:44.838307 8940 raft_consensus.cc:491] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:44.838351 8940 raft_consensus.cc:3058] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:44.839000 8940 raft_consensus.cc:513] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:44.839089 8940 leader_election.cc:304] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6537b4139ee49ebaaf570e251a9fb8a; no voters:
I20250815 02:08:44.839260 8940 leader_election.cc:290] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:44.839354 8945 raft_consensus.cc:2802] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:44.839493 8945 raft_consensus.cc:695] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 1 LEADER]: Becoming Leader. State: Replica: e6537b4139ee49ebaaf570e251a9fb8a, State: Running, Role: LEADER
I20250815 02:08:44.839582 8945 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:44.839592 8940 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:44.839926 8946 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } } }
I20250815 02:08:44.839931 8947 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6537b4139ee49ebaaf570e251a9fb8a. Latest consensus state: current_term: 1 leader_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } } }
I20250815 02:08:44.840054 8947 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:44.840382 8946 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:44.840503 8950 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:44.840947 8950 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:44.842723 8950 catalog_manager.cc:1349] Generated new cluster ID: 9ecf6bee843f4a6d9b5f1b3e4a88b10e
I20250815 02:08:44.842772 8950 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:44.854730 8950 catalog_manager.cc:1372] Generated new certificate authority record
I20250815 02:08:44.855237 8950 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:44.858227 8950 catalog_manager.cc:5955] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Generated new TSK 0
I20250815 02:08:44.858374 8950 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250815 02:08:44.860183 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.129:0
--local_ip_for_outbound_sockets=127.6.59.129
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42923
--builtin_ntp_servers=127.6.59.148:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:44.939507 8964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:44.939685 8964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:44.939720 8964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:44.941238 8964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:44.941316 8964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.129
I20250815 02:08:44.942836 8964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42923
--never_fsync=true
--heap_profile_path=/tmp/kudu.8964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.129
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:44.943071 8964 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:44.943292 8964 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:44.945744 8970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:44.945863 8964 server_base.cc:1047] running on GCE node
W20250815 02:08:44.945740 8969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:44.946056 8972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:44.946249 8964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:44.946527 8964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:44.947693 8964 hybrid_clock.cc:648] HybridClock initialized: now 1755223724947677 us; error 27 us; skew 500 ppm
I20250815 02:08:44.949048 8964 webserver.cc:480] Webserver started at http://127.6.59.129:41773/ using document root <none> and password file <none>
I20250815 02:08:44.949260 8964 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:44.949312 8964 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:44.949429 8964 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:44.950511 8964 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "5ba5c4b49af7483c92c08212584911d9"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.950878 8964 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "5ba5c4b49af7483c92c08212584911d9"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.952337 8964 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:44.953227 8978 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.953402 8964 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:44.953488 8964 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "5ba5c4b49af7483c92c08212584911d9"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:44.953550 8964 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:44.967489 8964 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:44.967748 8964 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:44.967880 8964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:44.968125 8964 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:44.968470 8964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:44.968521 8964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.968564 8964 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:44.968611 8964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:44.975669 8964 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.129:41291
I20250815 02:08:44.975749 9091 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.129:41291 every 8 connection(s)
I20250815 02:08:44.976065 8964 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250815 02:08:44.978621 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 8964
I20250815 02:08:44.978708 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250815 02:08:44.979794 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.130:0
--local_ip_for_outbound_sockets=127.6.59.130
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42923
--builtin_ntp_servers=127.6.59.148:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250815 02:08:44.982828 9092 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42923
I20250815 02:08:44.982940 9092 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:44.983121 9092 heartbeater.cc:507] Master 127.6.59.190:42923 requested a full tablet report, sending...
I20250815 02:08:44.983563 8904 ts_manager.cc:194] Registered new tserver with Master: 5ba5c4b49af7483c92c08212584911d9 (127.6.59.129:41291)
I20250815 02:08:44.984205 8904 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.129:33305
W20250815 02:08:45.062786 9095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:45.062955 9095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:45.062974 9095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:45.064303 9095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:45.064353 9095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.130
I20250815 02:08:45.065711 9095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42923
--never_fsync=true
--heap_profile_path=/tmp/kudu.9095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.130
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:45.065881 9095 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:45.066138 9095 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:45.068639 9101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:45.068620 9103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:45.068620 9100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:45.068981 9095 server_base.cc:1047] running on GCE node
I20250815 02:08:45.069156 9095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:45.069355 9095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:45.070483 9095 hybrid_clock.cc:648] HybridClock initialized: now 1755223725070466 us; error 34 us; skew 500 ppm
I20250815 02:08:45.071728 9095 webserver.cc:480] Webserver started at http://127.6.59.130:42073/ using document root <none> and password file <none>
I20250815 02:08:45.071923 9095 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:45.071964 9095 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:45.072077 9095 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:45.072932 9095 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "18b448977b9b455d93c91367f279b227"
format_stamp: "Formatted at 2025-08-15 02:08:45 on dist-test-slave-wq2q"
I20250815 02:08:45.073225 9095 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "18b448977b9b455d93c91367f279b227"
format_stamp: "Formatted at 2025-08-15 02:08:45 on dist-test-slave-wq2q"
I20250815 02:08:45.074412 9095 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250815 02:08:45.075085 9109 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:45.075248 9095 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:45.075309 9095 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "18b448977b9b455d93c91367f279b227"
format_stamp: "Formatted at 2025-08-15 02:08:45 on dist-test-slave-wq2q"
I20250815 02:08:45.075371 9095 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:45.084758 9095 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:45.084966 9095 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:45.085068 9095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:45.085258 9095 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:45.085541 9095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:45.085572 9095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:45.085606 9095 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:45.085646 9095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:45.090854 9095 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.130:33815
I20250815 02:08:45.090924 9222 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.130:33815 every 8 connection(s)
I20250815 02:08:45.091193 9095 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250815 02:08:45.094460 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 9095
I20250815 02:08:45.094525 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250815 02:08:45.095520 9223 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42923
I20250815 02:08:45.095606 9223 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:45.095656 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.131:0
--local_ip_for_outbound_sockets=127.6.59.131
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42923
--builtin_ntp_servers=127.6.59.148:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250815 02:08:45.095816 9223 heartbeater.cc:507] Master 127.6.59.190:42923 requested a full tablet report, sending...
I20250815 02:08:45.096163 8904 ts_manager.cc:194] Registered new tserver with Master: 18b448977b9b455d93c91367f279b227 (127.6.59.130:33815)
I20250815 02:08:45.096499 8904 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.130:54223
W20250815 02:08:45.172706 9226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:45.172883 9226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:45.172915 9226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:45.174345 9226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:45.174413 9226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.131
I20250815 02:08:45.175840 9226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42923
--never_fsync=true
--heap_profile_path=/tmp/kudu.9226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.131
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:45.176086 9226 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:45.176301 9226 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:45.178740 9231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:45.178750 9232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:45.178740 9234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:45.178821 9226 server_base.cc:1047] running on GCE node
I20250815 02:08:45.179200 9226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:45.179370 9226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:45.180507 9226 hybrid_clock.cc:648] HybridClock initialized: now 1755223725180496 us; error 28 us; skew 500 ppm
I20250815 02:08:45.181643 9226 webserver.cc:480] Webserver started at http://127.6.59.131:34203/ using document root <none> and password file <none>
I20250815 02:08:45.181850 9226 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:45.181906 9226 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:45.182057 9226 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:45.182904 9226 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "144c0179dfbb4ff499614d1b6d3cab01"
format_stamp: "Formatted at 2025-08-15 02:08:45 on dist-test-slave-wq2q"
I20250815 02:08:45.183214 9226 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "144c0179dfbb4ff499614d1b6d3cab01"
format_stamp: "Formatted at 2025-08-15 02:08:45 on dist-test-slave-wq2q"
I20250815 02:08:45.184401 9226 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:45.185083 9240 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:45.185228 9226 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:45.185298 9226 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "144c0179dfbb4ff499614d1b6d3cab01"
format_stamp: "Formatted at 2025-08-15 02:08:45 on dist-test-slave-wq2q"
I20250815 02:08:45.185354 9226 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:45.199921 9226 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:45.200178 9226 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:45.200291 9226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:45.200516 9226 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:45.200840 9226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:45.200881 9226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:45.200917 9226 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:45.200945 9226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:45.206447 9226 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.131:37121
I20250815 02:08:45.206506 9353 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.131:37121 every 8 connection(s)
I20250815 02:08:45.206799 9226 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250815 02:08:45.209894 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 9226
I20250815 02:08:45.209997 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250815 02:08:45.210970 9354 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42923
I20250815 02:08:45.211071 9354 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:45.211270 9354 heartbeater.cc:507] Master 127.6.59.190:42923 requested a full tablet report, sending...
I20250815 02:08:45.211614 8904 ts_manager.cc:194] Registered new tserver with Master: 144c0179dfbb4ff499614d1b6d3cab01 (127.6.59.131:37121)
I20250815 02:08:45.211973 8904 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.131:49345
I20250815 02:08:45.222028 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:45.227720 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:45.227778 6382 test_util.cc:276] Using random seed: -559628542
I20250815 02:08:45.233338 8904 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45684:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250815 02:08:45.239069 9026 tablet_service.cc:1468] Processing CreateTablet for tablet 3ccfe0b0dfe842faa39d995d7b356dea (DEFAULT_TABLE table=TestTable [id=9b11908d075c4189b051b332b32359af]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:45.239372 9026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ccfe0b0dfe842faa39d995d7b356dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:45.241557 9369 tablet_bootstrap.cc:492] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Bootstrap starting.
I20250815 02:08:45.242179 9369 tablet_bootstrap.cc:654] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:45.242441 9369 log.cc:826] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:45.243046 9369 tablet_bootstrap.cc:492] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: No bootstrap required, opened a new log
I20250815 02:08:45.243096 9369 ts_tablet_manager.cc:1397] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:45.244206 9369 raft_consensus.cc:357] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } }
I20250815 02:08:45.244316 9369 raft_consensus.cc:383] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:45.244336 9369 raft_consensus.cc:738] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ba5c4b49af7483c92c08212584911d9, State: Initialized, Role: FOLLOWER
I20250815 02:08:45.244411 9369 consensus_queue.cc:260] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } }
I20250815 02:08:45.244472 9369 raft_consensus.cc:397] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:45.244503 9369 raft_consensus.cc:491] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:45.244549 9369 raft_consensus.cc:3058] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:45.245056 9369 raft_consensus.cc:513] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } }
I20250815 02:08:45.245147 9369 leader_election.cc:304] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ba5c4b49af7483c92c08212584911d9; no voters:
I20250815 02:08:45.245316 9369 leader_election.cc:290] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:45.245414 9371 raft_consensus.cc:2802] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:45.245496 9369 ts_tablet_manager.cc:1428] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250815 02:08:45.245517 9371 raft_consensus.cc:695] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 1 LEADER]: Becoming Leader. State: Replica: 5ba5c4b49af7483c92c08212584911d9, State: Running, Role: LEADER
I20250815 02:08:45.245527 9092 heartbeater.cc:499] Master 127.6.59.190:42923 was elected leader, sending a full tablet report...
I20250815 02:08:45.245664 9371 consensus_queue.cc:237] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } }
I20250815 02:08:45.246479 8904 catalog_manager.cc:5582] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ba5c4b49af7483c92c08212584911d9 (127.6.59.129). New cstate: current_term: 1 leader_uuid: "5ba5c4b49af7483c92c08212584911d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } health_report { overall_health: HEALTHY } } }
I20250815 02:08:45.259371 6382 test_util.cc:276] Using random seed: -559596962
I20250815 02:08:45.263866 8903 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45698:
name: "TestTable1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250815 02:08:45.268671 9157 tablet_service.cc:1468] Processing CreateTablet for tablet 21788a4474f044e1b01424e7e6bb0687 (DEFAULT_TABLE table=TestTable1 [id=a2358edd92184b6d909cf48bc1d01c25]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:45.268939 9157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21788a4474f044e1b01424e7e6bb0687. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:45.270888 9389 tablet_bootstrap.cc:492] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Bootstrap starting.
I20250815 02:08:45.271421 9389 tablet_bootstrap.cc:654] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:45.271632 9389 log.cc:826] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:45.272161 9389 tablet_bootstrap.cc:492] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: No bootstrap required, opened a new log
I20250815 02:08:45.272210 9389 ts_tablet_manager.cc:1397] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:45.273161 9389 raft_consensus.cc:357] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } }
I20250815 02:08:45.273270 9389 raft_consensus.cc:383] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:45.273289 9389 raft_consensus.cc:738] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b448977b9b455d93c91367f279b227, State: Initialized, Role: FOLLOWER
I20250815 02:08:45.273365 9389 consensus_queue.cc:260] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } }
I20250815 02:08:45.273430 9389 raft_consensus.cc:397] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:45.273460 9389 raft_consensus.cc:491] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:45.273505 9389 raft_consensus.cc:3058] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:45.274099 9389 raft_consensus.cc:513] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } }
I20250815 02:08:45.274191 9389 leader_election.cc:304] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18b448977b9b455d93c91367f279b227; no voters:
I20250815 02:08:45.274328 9389 leader_election.cc:290] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:45.274399 9391 raft_consensus.cc:2802] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:45.274510 9389 ts_tablet_manager.cc:1428] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250815 02:08:45.274538 9223 heartbeater.cc:499] Master 127.6.59.190:42923 was elected leader, sending a full tablet report...
I20250815 02:08:45.274576 9391 raft_consensus.cc:695] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 1 LEADER]: Becoming Leader. State: Replica: 18b448977b9b455d93c91367f279b227, State: Running, Role: LEADER
I20250815 02:08:45.274695 9391 consensus_queue.cc:237] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } }
I20250815 02:08:45.275352 8903 catalog_manager.cc:5582] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18b448977b9b455d93c91367f279b227 (127.6.59.130). New cstate: current_term: 1 leader_uuid: "18b448977b9b455d93c91367f279b227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } health_report { overall_health: HEALTHY } } }
I20250815 02:08:45.288730 6382 test_util.cc:276] Using random seed: -559567601
I20250815 02:08:45.292941 8903 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45704:
name: "TestTable2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250815 02:08:45.297379 9288 tablet_service.cc:1468] Processing CreateTablet for tablet a38410e3c42a4394849a2a575857fc2e (DEFAULT_TABLE table=TestTable2 [id=a0164f1781af4a69ba4dfc408d288cff]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:45.297703 9288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38410e3c42a4394849a2a575857fc2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:45.299645 9410 tablet_bootstrap.cc:492] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Bootstrap starting.
I20250815 02:08:45.300298 9410 tablet_bootstrap.cc:654] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:45.300539 9410 log.cc:826] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:45.301095 9410 tablet_bootstrap.cc:492] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: No bootstrap required, opened a new log
I20250815 02:08:45.301146 9410 ts_tablet_manager.cc:1397] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:45.302147 9410 raft_consensus.cc:357] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 37121 } }
I20250815 02:08:45.302253 9410 raft_consensus.cc:383] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:45.302273 9410 raft_consensus.cc:738] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c0179dfbb4ff499614d1b6d3cab01, State: Initialized, Role: FOLLOWER
I20250815 02:08:45.302361 9410 consensus_queue.cc:260] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 37121 } }
I20250815 02:08:45.302414 9410 raft_consensus.cc:397] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:45.302445 9410 raft_consensus.cc:491] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:45.302489 9410 raft_consensus.cc:3058] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:45.303136 9410 raft_consensus.cc:513] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 37121 } }
I20250815 02:08:45.303234 9410 leader_election.cc:304] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 144c0179dfbb4ff499614d1b6d3cab01; no voters:
I20250815 02:08:45.303385 9410 leader_election.cc:290] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:45.303448 9412 raft_consensus.cc:2802] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:45.303597 9354 heartbeater.cc:499] Master 127.6.59.190:42923 was elected leader, sending a full tablet report...
I20250815 02:08:45.303635 9412 raft_consensus.cc:695] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 1 LEADER]: Becoming Leader. State: Replica: 144c0179dfbb4ff499614d1b6d3cab01, State: Running, Role: LEADER
I20250815 02:08:45.303740 9412 consensus_queue.cc:237] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 37121 } }
I20250815 02:08:45.303784 9410 ts_tablet_manager.cc:1428] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250815 02:08:45.304379 8903 catalog_manager.cc:5582] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 144c0179dfbb4ff499614d1b6d3cab01 (127.6.59.131). New cstate: current_term: 1 leader_uuid: "144c0179dfbb4ff499614d1b6d3cab01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 37121 } health_report { overall_health: HEALTHY } } }
I20250815 02:08:45.317845 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 8873
I20250815 02:08:45.645009 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 8964
I20250815 02:08:45.649443 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 9095
I20250815 02:08:45.653595 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 9226
I20250815 02:08:45.658156 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:42923
--webserver_interface=127.6.59.190
--webserver_port=45445
--builtin_ntp_servers=127.6.59.148:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:42923 with env {}
W20250815 02:08:45.732570 9470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:45.732743 9470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:45.732775 9470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:45.734133 9470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:45.734171 9470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:45.734189 9470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:45.734210 9470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:45.735558 9470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:42923
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:42923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=45445
--never_fsync=true
--heap_profile_path=/tmp/kudu.9470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:45.735795 9470 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:45.735996 9470 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:45.738188 9476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:45.738200 9475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:45.738204 9478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:45.738245 9470 server_base.cc:1047] running on GCE node
I20250815 02:08:45.738564 9470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:45.738785 9470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:45.739916 9470 hybrid_clock.cc:648] HybridClock initialized: now 1755223725739899 us; error 34 us; skew 500 ppm
I20250815 02:08:45.741044 9470 webserver.cc:480] Webserver started at http://127.6.59.190:45445/ using document root <none> and password file <none>
I20250815 02:08:45.741250 9470 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:45.741295 9470 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:45.742515 9470 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:45.743125 9484 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:45.743283 9470 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:45.743353 9470 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
uuid: "e6537b4139ee49ebaaf570e251a9fb8a"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:45.743602 9470 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:45.768292 9470 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:45.768608 9470 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:45.768734 9470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:45.772446 9470 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:42923
I20250815 02:08:45.772492 9536 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:42923 every 8 connection(s)
I20250815 02:08:45.772827 9470 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250815 02:08:45.773881 9537 sys_catalog.cc:263] Verifying existing consensus state
I20250815 02:08:45.774524 9537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Bootstrap starting.
I20250815 02:08:45.775822 9537 log.cc:826] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:45.777902 9537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=13 ignored=0} mutations{seen=10 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:45.778187 9537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Bootstrap complete.
I20250815 02:08:45.779417 9537 raft_consensus.cc:357] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:45.779587 9537 raft_consensus.cc:738] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6537b4139ee49ebaaf570e251a9fb8a, State: Initialized, Role: FOLLOWER
I20250815 02:08:45.779732 9537 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:45.779793 9537 raft_consensus.cc:397] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:45.779821 9537 raft_consensus.cc:491] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:45.779873 9537 raft_consensus.cc:3058] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 2 FOLLOWER]: Advancing to term 3
I20250815 02:08:45.780686 9537 raft_consensus.cc:513] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:45.780773 9537 leader_election.cc:304] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6537b4139ee49ebaaf570e251a9fb8a; no voters:
I20250815 02:08:45.780920 9537 leader_election.cc:290] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [CANDIDATE]: Term 3 election: Requested vote from peers
I20250815 02:08:45.780984 9540 raft_consensus.cc:2802] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 3 FOLLOWER]: Leader election won for term 3
I20250815 02:08:45.781153 9540 raft_consensus.cc:695] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 3 LEADER]: Becoming Leader. State: Replica: e6537b4139ee49ebaaf570e251a9fb8a, State: Running, Role: LEADER
I20250815 02:08:45.781137 9537 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:45.781265 9540 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:45.781541 9541 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } } }
I20250815 02:08:45.781657 9541 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:45.781550 9542 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6537b4139ee49ebaaf570e251a9fb8a. Latest consensus state: current_term: 3 leader_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } } }
I20250815 02:08:45.781953 9542 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:45.782145 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 9470
I20250815 02:08:45.782405 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.129:41291
--local_ip_for_outbound_sockets=127.6.59.129
--tserver_master_addrs=127.6.59.190:42923
--webserver_port=41773
--webserver_interface=127.6.59.129
--builtin_ntp_servers=127.6.59.148:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250815 02:08:45.783124 9556 catalog_manager.cc:1261] Loaded cluster ID: 9ecf6bee843f4a6d9b5f1b3e4a88b10e
I20250815 02:08:45.783174 9556 catalog_manager.cc:1554] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: loading cluster ID for follower catalog manager: success
I20250815 02:08:45.784416 9556 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: acquiring CA information for follower catalog manager: success
I20250815 02:08:45.784736 9556 catalog_manager.cc:1604] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250815 02:08:45.784930 9546 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:45.785163 9546 catalog_manager.cc:671] Loaded metadata for table TestTable [id=34c90744fdf344679c85549e55614c75]
I20250815 02:08:45.785241 9546 catalog_manager.cc:671] Loaded metadata for table TestTable2 [id=a0164f1781af4a69ba4dfc408d288cff]
I20250815 02:08:45.785279 9546 catalog_manager.cc:671] Loaded metadata for table TestTable1 [id=a0771eca8d9e4fe0b8c7502d8aaf1491]
I20250815 02:08:45.785404 9546 tablet_loader.cc:96] loaded metadata for tablet 21788a4474f044e1b01424e7e6bb0687 (table TestTable1 [id=a0771eca8d9e4fe0b8c7502d8aaf1491])
I20250815 02:08:45.785456 9546 tablet_loader.cc:96] loaded metadata for tablet 3ccfe0b0dfe842faa39d995d7b356dea (table TestTable [id=34c90744fdf344679c85549e55614c75])
I20250815 02:08:45.785485 9546 tablet_loader.cc:96] loaded metadata for tablet a38410e3c42a4394849a2a575857fc2e (table TestTable2 [id=a0164f1781af4a69ba4dfc408d288cff])
I20250815 02:08:45.785511 9546 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:45.785567 9546 catalog_manager.cc:1261] Loaded cluster ID: 9ecf6bee843f4a6d9b5f1b3e4a88b10e
I20250815 02:08:45.785590 9546 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:45.785759 9546 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:45.785825 9546 catalog_manager.cc:5966] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Loaded TSK: 0
I20250815 02:08:45.785929 9546 catalog_manager.cc:1516] Initializing in-progress tserver states...
W20250815 02:08:45.858685 9558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:45.858847 9558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:45.858876 9558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:45.860224 9558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:45.860283 9558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.129
I20250815 02:08:45.861721 9558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.129:41291
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.59.129
--webserver_port=41773
--tserver_master_addrs=127.6.59.190:42923
--never_fsync=true
--heap_profile_path=/tmp/kudu.9558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.129
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:45.861943 9558 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:45.862202 9558 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:45.864631 9564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:45.864704 9563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:45.864665 9566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:45.864748 9558 server_base.cc:1047] running on GCE node
I20250815 02:08:45.865029 9558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:45.865259 9558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:45.866427 9558 hybrid_clock.cc:648] HybridClock initialized: now 1755223725866412 us; error 31 us; skew 500 ppm
I20250815 02:08:45.867784 9558 webserver.cc:480] Webserver started at http://127.6.59.129:41773/ using document root <none> and password file <none>
I20250815 02:08:45.867991 9558 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:45.868062 9558 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:45.869537 9558 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:45.870357 9572 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:45.870527 9558 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:45.870611 9558 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "5ba5c4b49af7483c92c08212584911d9"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:45.870954 9558 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:45.883273 9558 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:45.883520 9558 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:45.883641 9558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:45.883874 9558 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:45.884332 9579 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250815 02:08:45.885198 9558 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250815 02:08:45.885255 9558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:45.885303 9558 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250815 02:08:45.886076 9558 ts_tablet_manager.cc:610] Registered 1 tablets
I20250815 02:08:45.886134 9558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:45.886152 9579 tablet_bootstrap.cc:492] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Bootstrap starting.
I20250815 02:08:45.893076 9558 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.129:41291
I20250815 02:08:45.893478 9558 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250815 02:08:45.894200 9686 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.129:41291 every 8 connection(s)
I20250815 02:08:45.896044 9579 log.cc:826] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:45.896862 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 9558
I20250815 02:08:45.897110 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.130:33815
--local_ip_for_outbound_sockets=127.6.59.130
--tserver_master_addrs=127.6.59.190:42923
--webserver_port=42073
--webserver_interface=127.6.59.130
--builtin_ntp_servers=127.6.59.148:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250815 02:08:45.899746 9579 tablet_bootstrap.cc:492] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:45.900000 9579 tablet_bootstrap.cc:492] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Bootstrap complete.
I20250815 02:08:45.900815 9579 ts_tablet_manager.cc:1397] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.005s
I20250815 02:08:45.902009 9579 raft_consensus.cc:357] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } }
I20250815 02:08:45.902309 9579 raft_consensus.cc:738] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ba5c4b49af7483c92c08212584911d9, State: Initialized, Role: FOLLOWER
I20250815 02:08:45.902455 9579 consensus_queue.cc:260] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } }
I20250815 02:08:45.902522 9579 raft_consensus.cc:397] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:45.902606 9579 raft_consensus.cc:491] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:45.902678 9579 raft_consensus.cc:3058] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:45.904019 9579 raft_consensus.cc:513] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } }
I20250815 02:08:45.904201 9579 leader_election.cc:304] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ba5c4b49af7483c92c08212584911d9; no voters:
I20250815 02:08:45.904386 9579 leader_election.cc:290] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20250815 02:08:45.904441 9693 raft_consensus.cc:2802] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 2 FOLLOWER]: Leader election won for term 2
I20250815 02:08:45.904645 9693 raft_consensus.cc:695] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 2 LEADER]: Becoming Leader. State: Replica: 5ba5c4b49af7483c92c08212584911d9, State: Running, Role: LEADER
I20250815 02:08:45.904649 9579 ts_tablet_manager.cc:1428] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20250815 02:08:45.904809 9693 consensus_queue.cc:237] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } }
I20250815 02:08:45.921150 9687 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42923
I20250815 02:08:45.921280 9687 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:45.921492 9687 heartbeater.cc:507] Master 127.6.59.190:42923 requested a full tablet report, sending...
I20250815 02:08:45.922206 9501 ts_manager.cc:194] Registered new tserver with Master: 5ba5c4b49af7483c92c08212584911d9 (127.6.59.129:41291)
I20250815 02:08:45.922351 9501 catalog_manager.cc:5582] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 reported cstate change: term changed from 0 to 2, leader changed from <none> to 5ba5c4b49af7483c92c08212584911d9 (127.6.59.129), VOTER 5ba5c4b49af7483c92c08212584911d9 (127.6.59.129) added. New cstate: current_term: 2 leader_uuid: "5ba5c4b49af7483c92c08212584911d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } health_report { overall_health: HEALTHY } } }
W20250815 02:08:45.925300 9501 catalog_manager.cc:5260] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 3ccfe0b0dfe842faa39d995d7b356dea with cas_config_opid_index -1: no extra replica candidate found for tablet 3ccfe0b0dfe842faa39d995d7b356dea (table TestTable [id=34c90744fdf344679c85549e55614c75]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (1) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
I20250815 02:08:45.926307 9501 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.129:49845
I20250815 02:08:45.927139 9687 heartbeater.cc:499] Master 127.6.59.190:42923 was elected leader, sending a full tablet report...
W20250815 02:08:45.981153 9692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:45.981310 9692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:45.981339 9692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:45.982856 9692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:45.982918 9692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.130
I20250815 02:08:45.984354 9692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.130:33815
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.59.130
--webserver_port=42073
--tserver_master_addrs=127.6.59.190:42923
--never_fsync=true
--heap_profile_path=/tmp/kudu.9692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.130
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:45.984568 9692 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:45.984768 9692 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:45.987306 9706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:45.987289 9692 server_base.cc:1047] running on GCE node
W20250815 02:08:45.987282 9704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:45.987265 9703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:45.987627 9692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:45.987831 9692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:45.988991 9692 hybrid_clock.cc:648] HybridClock initialized: now 1755223725988977 us; error 29 us; skew 500 ppm
I20250815 02:08:45.990248 9692 webserver.cc:480] Webserver started at http://127.6.59.130:42073/ using document root <none> and password file <none>
I20250815 02:08:45.990434 9692 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:45.990490 9692 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:45.991621 9692 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20250815 02:08:45.992221 9712 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:45.992383 9692 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:45.992448 9692 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "18b448977b9b455d93c91367f279b227"
format_stamp: "Formatted at 2025-08-15 02:08:45 on dist-test-slave-wq2q"
I20250815 02:08:45.992707 9692 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:46.000141 9692 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:46.000321 9692 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:46.000401 9692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:46.000562 9692 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:46.000905 9719 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250815 02:08:46.001667 9692 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250815 02:08:46.001700 9692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:46.001721 9692 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250815 02:08:46.002292 9692 ts_tablet_manager.cc:610] Registered 1 tablets
I20250815 02:08:46.002323 9692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:46.002409 9719 tablet_bootstrap.cc:492] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Bootstrap starting.
I20250815 02:08:46.009073 9692 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.130:33815
I20250815 02:08:46.009125 9826 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.130:33815 every 8 connection(s)
I20250815 02:08:46.009557 9692 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250815 02:08:46.012812 9719 log.cc:826] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:46.014092 9827 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42923
I20250815 02:08:46.014175 9827 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:46.014357 9827 heartbeater.cc:507] Master 127.6.59.190:42923 requested a full tablet report, sending...
I20250815 02:08:46.014832 9501 ts_manager.cc:194] Registered new tserver with Master: 18b448977b9b455d93c91367f279b227 (127.6.59.130:33815)
I20250815 02:08:46.015237 9501 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.130:34633
I20250815 02:08:46.015731 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 9692
I20250815 02:08:46.016019 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.131:37121
--local_ip_for_outbound_sockets=127.6.59.131
--tserver_master_addrs=127.6.59.190:42923
--webserver_port=34203
--webserver_interface=127.6.59.131
--builtin_ntp_servers=127.6.59.148:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250815 02:08:46.017237 9719 tablet_bootstrap.cc:492] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=650 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:46.017539 9719 tablet_bootstrap.cc:492] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Bootstrap complete.
I20250815 02:08:46.018393 9719 ts_tablet_manager.cc:1397] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.015s
I20250815 02:08:46.019150 9719 raft_consensus.cc:357] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } }
I20250815 02:08:46.019323 9719 raft_consensus.cc:738] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b448977b9b455d93c91367f279b227, State: Initialized, Role: FOLLOWER
I20250815 02:08:46.019409 9719 consensus_queue.cc:260] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 1.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } }
I20250815 02:08:46.019480 9719 raft_consensus.cc:397] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:46.019526 9719 raft_consensus.cc:491] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:46.019579 9719 raft_consensus.cc:3058] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:46.020979 9719 raft_consensus.cc:513] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } }
I20250815 02:08:46.021109 9719 leader_election.cc:304] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18b448977b9b455d93c91367f279b227; no voters:
I20250815 02:08:46.021283 9719 leader_election.cc:290] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [CANDIDATE]: Term 2 election: Requested vote from peers
I20250815 02:08:46.021342 9832 raft_consensus.cc:2802] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 2 FOLLOWER]: Leader election won for term 2
I20250815 02:08:46.021492 9832 raft_consensus.cc:695] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 2 LEADER]: Becoming Leader. State: Replica: 18b448977b9b455d93c91367f279b227, State: Running, Role: LEADER
I20250815 02:08:46.021507 9719 ts_tablet_manager.cc:1428] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20250815 02:08:46.021714 9832 consensus_queue.cc:237] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 14, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } }
I20250815 02:08:46.021591 9827 heartbeater.cc:499] Master 127.6.59.190:42923 was elected leader, sending a full tablet report...
I20250815 02:08:46.022358 9501 catalog_manager.cc:5582] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 reported cstate change: term changed from 0 to 2, leader changed from <none> to 18b448977b9b455d93c91367f279b227 (127.6.59.130), VOTER 18b448977b9b455d93c91367f279b227 (127.6.59.130) added. New cstate: current_term: 2 leader_uuid: "18b448977b9b455d93c91367f279b227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } health_report { overall_health: HEALTHY } } }
I20250815 02:08:46.029426 9781 consensus_queue.cc:237] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 2.15, Last appended by leader: 14, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } } peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: NON_VOTER last_known_addr { host: "127.6.59.129" port: 41291 } attrs { promote: true } }
I20250815 02:08:46.029955 9641 consensus_queue.cc:237] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } } peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: NON_VOTER last_known_addr { host: "127.6.59.130" port: 33815 } attrs { promote: true } }
I20250815 02:08:46.030380 9832 raft_consensus.cc:2953] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 2 LEADER]: Committing config change with OpId 2.16: config changed from index -1 to 16, NON_VOTER 5ba5c4b49af7483c92c08212584911d9 (127.6.59.129) added. New config: { opid_index: 16 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } } peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: NON_VOTER last_known_addr { host: "127.6.59.129" port: 41291 } attrs { promote: true } } }
I20250815 02:08:46.030457 9695 raft_consensus.cc:2953] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index -1 to 11, NON_VOTER 18b448977b9b455d93c91367f279b227 (127.6.59.130) added. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } } peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: NON_VOTER last_known_addr { host: "127.6.59.130" port: 33815 } attrs { promote: true } } }
I20250815 02:08:46.031482 9487 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 21788a4474f044e1b01424e7e6bb0687 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20250815 02:08:46.031946 9501 catalog_manager.cc:5582] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 reported cstate change: config changed from index -1 to 11, NON_VOTER 18b448977b9b455d93c91367f279b227 (127.6.59.130) added. New cstate: current_term: 2 leader_uuid: "5ba5c4b49af7483c92c08212584911d9" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: NON_VOTER last_known_addr { host: "127.6.59.130" port: 33815 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250815 02:08:46.032330 9500 catalog_manager.cc:5582] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 reported cstate change: config changed from index -1 to 16, NON_VOTER 5ba5c4b49af7483c92c08212584911d9 (127.6.59.129) added. New cstate: current_term: 2 leader_uuid: "18b448977b9b455d93c91367f279b227" committed_config { opid_index: 16 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: NON_VOTER last_known_addr { host: "127.6.59.129" port: 41291 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250815 02:08:46.032517 9486 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 3ccfe0b0dfe842faa39d995d7b356dea with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 3)
W20250815 02:08:46.033762 9714 consensus_peers.cc:489] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 -> Peer 5ba5c4b49af7483c92c08212584911d9 (127.6.59.129:41291): Couldn't send request to peer 5ba5c4b49af7483c92c08212584911d9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21788a4474f044e1b01424e7e6bb0687. This is attempt 1: this message will repeat every 5th retry.
W20250815 02:08:46.035310 9575 consensus_peers.cc:489] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 -> Peer 18b448977b9b455d93c91367f279b227 (127.6.59.130:33815): Couldn't send request to peer 18b448977b9b455d93c91367f279b227. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ccfe0b0dfe842faa39d995d7b356dea. This is attempt 1: this message will repeat every 5th retry.
W20250815 02:08:46.098079 9831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:46.098232 9831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:46.098251 9831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:46.099660 9831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:46.099711 9831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.131
I20250815 02:08:46.101137 9831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.131:37121
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.59.131
--webserver_port=34203
--tserver_master_addrs=127.6.59.190:42923
--never_fsync=true
--heap_profile_path=/tmp/kudu.9831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.131
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:46.101356 9831 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:46.101572 9831 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:46.104156 9845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:46.104192 9846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:46.104195 9848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:46.104349 9831 server_base.cc:1047] running on GCE node
I20250815 02:08:46.104504 9831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:46.104708 9831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:46.105851 9831 hybrid_clock.cc:648] HybridClock initialized: now 1755223726105835 us; error 32 us; skew 500 ppm
I20250815 02:08:46.107021 9831 webserver.cc:480] Webserver started at http://127.6.59.131:34203/ using document root <none> and password file <none>
I20250815 02:08:46.107256 9831 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:46.107329 9831 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:46.108981 9831 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20250815 02:08:46.109760 9854 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:46.109952 9831 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:46.110054 9831 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "144c0179dfbb4ff499614d1b6d3cab01"
format_stamp: "Formatted at 2025-08-15 02:08:45 on dist-test-slave-wq2q"
I20250815 02:08:46.110415 9831 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:46.128419 9831 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:46.128665 9831 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:46.128752 9831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:46.128937 9831 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:46.129344 9861 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250815 02:08:46.130232 9831 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250815 02:08:46.130273 9831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:46.130297 9831 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250815 02:08:46.130829 9831 ts_tablet_manager.cc:610] Registered 1 tablets
I20250815 02:08:46.130867 9831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:46.130935 9861 tablet_bootstrap.cc:492] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Bootstrap starting.
I20250815 02:08:46.136790 9831 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.131:37121
I20250815 02:08:46.136852 9968 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.131:37121 every 8 connection(s)
I20250815 02:08:46.137136 9831 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250815 02:08:46.141124 9861 log.cc:826] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:46.141772 9969 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42923
I20250815 02:08:46.141863 9969 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:46.142077 9969 heartbeater.cc:507] Master 127.6.59.190:42923 requested a full tablet report, sending...
I20250815 02:08:46.142585 9500 ts_manager.cc:194] Registered new tserver with Master: 144c0179dfbb4ff499614d1b6d3cab01 (127.6.59.131:37121)
I20250815 02:08:46.142894 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 9831
I20250815 02:08:46.143443 9500 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.131:58235
I20250815 02:08:46.145722 9641 consensus_queue.cc:237] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } } peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: NON_VOTER last_known_addr { host: "127.6.59.130" port: 33815 } attrs { promote: true } } peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: NON_VOTER last_known_addr { host: "127.6.59.131" port: 37121 } attrs { promote: true } }
I20250815 02:08:46.145840 9861 tablet_bootstrap.cc:492] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=650 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:46.146181 9861 tablet_bootstrap.cc:492] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Bootstrap complete.
I20250815 02:08:46.146265 9695 raft_consensus.cc:2953] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 [term 2 LEADER]: Committing config change with OpId 2.12: config changed from index 11 to 12, NON_VOTER 144c0179dfbb4ff499614d1b6d3cab01 (127.6.59.131) added. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } } peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: NON_VOTER last_known_addr { host: "127.6.59.130" port: 33815 } attrs { promote: true } } peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: NON_VOTER last_known_addr { host: "127.6.59.131" port: 37121 } attrs { promote: true } } }
I20250815 02:08:46.147120 9861 ts_tablet_manager.cc:1397] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.012s
W20250815 02:08:46.147233 9575 consensus_peers.cc:489] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 -> Peer 18b448977b9b455d93c91367f279b227 (127.6.59.130:33815): Couldn't send request to peer 18b448977b9b455d93c91367f279b227. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ccfe0b0dfe842faa39d995d7b356dea. This is attempt 1: this message will repeat every 5th retry.
I20250815 02:08:46.147307 9486 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 3ccfe0b0dfe842faa39d995d7b356dea with cas_config_opid_index 11: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 3)
I20250815 02:08:46.147430 9500 catalog_manager.cc:5582] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 reported cstate change: config changed from index 11 to 12, NON_VOTER 144c0179dfbb4ff499614d1b6d3cab01 (127.6.59.131) added. New cstate: current_term: 2 leader_uuid: "5ba5c4b49af7483c92c08212584911d9" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 41291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: NON_VOTER last_known_addr { host: "127.6.59.130" port: 33815 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: NON_VOTER last_known_addr { host: "127.6.59.131" port: 37121 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250815 02:08:46.148290 9861 raft_consensus.cc:357] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 37121 } }
I20250815 02:08:46.148561 9861 raft_consensus.cc:738] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c0179dfbb4ff499614d1b6d3cab01, State: Initialized, Role: FOLLOWER
I20250815 02:08:46.148705 9861 consensus_queue.cc:260] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 1.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 37121 } }
I20250815 02:08:46.148772 9861 raft_consensus.cc:397] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:46.148808 9861 raft_consensus.cc:491] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:46.148871 9861 raft_consensus.cc:3058] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:46.149098 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:46.149708 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:46.150074 9861 raft_consensus.cc:513] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 37121 } }
I20250815 02:08:46.150183 9861 leader_election.cc:304] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 144c0179dfbb4ff499614d1b6d3cab01; no voters:
I20250815 02:08:46.150264 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 5ba5c4b49af7483c92c08212584911d9 to finish bootstrapping
I20250815 02:08:46.150333 9861 leader_election.cc:290] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [CANDIDATE]: Term 2 election: Requested vote from peers
I20250815 02:08:46.150496 9976 raft_consensus.cc:2802] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 2 FOLLOWER]: Leader election won for term 2
I20250815 02:08:46.150524 9861 ts_tablet_manager.cc:1428] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20250815 02:08:46.150650 9969 heartbeater.cc:499] Master 127.6.59.190:42923 was elected leader, sending a full tablet report...
I20250815 02:08:46.150602 9976 raft_consensus.cc:695] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [term 2 LEADER]: Becoming Leader. State: Replica: 144c0179dfbb4ff499614d1b6d3cab01, State: Running, Role: LEADER
I20250815 02:08:46.150911 9976 consensus_queue.cc:237] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 14, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 37121 } }
W20250815 02:08:46.151345 9576 consensus_peers.cc:489] T 3ccfe0b0dfe842faa39d995d7b356dea P 5ba5c4b49af7483c92c08212584911d9 -> Peer 144c0179dfbb4ff499614d1b6d3cab01 (127.6.59.131:37121): Couldn't send request to peer 144c0179dfbb4ff499614d1b6d3cab01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ccfe0b0dfe842faa39d995d7b356dea. This is attempt 1: this message will repeat every 5th retry.
I20250815 02:08:46.151429 9500 catalog_manager.cc:5582] T a38410e3c42a4394849a2a575857fc2e P 144c0179dfbb4ff499614d1b6d3cab01 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "144c0179dfbb4ff499614d1b6d3cab01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 37121 } health_report { overall_health: HEALTHY } } }
I20250815 02:08:46.153676 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 18b448977b9b455d93c91367f279b227 to finish bootstrapping
I20250815 02:08:46.156507 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 144c0179dfbb4ff499614d1b6d3cab01 to finish bootstrapping
I20250815 02:08:46.166842 9781 consensus_queue.cc:237] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 2.16, Last appended by leader: 14, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } } peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: NON_VOTER last_known_addr { host: "127.6.59.129" port: 41291 } attrs { promote: true } } peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: NON_VOTER last_known_addr { host: "127.6.59.131" port: 37121 } attrs { promote: true } }
I20250815 02:08:46.167263 9832 raft_consensus.cc:2953] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 [term 2 LEADER]: Committing config change with OpId 2.17: config changed from index 16 to 17, NON_VOTER 144c0179dfbb4ff499614d1b6d3cab01 (127.6.59.131) added. New config: { opid_index: 17 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } } peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: NON_VOTER last_known_addr { host: "127.6.59.129" port: 41291 } attrs { promote: true } } peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: NON_VOTER last_known_addr { host: "127.6.59.131" port: 37121 } attrs { promote: true } } }
W20250815 02:08:46.167644 9714 consensus_peers.cc:489] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 -> Peer 5ba5c4b49af7483c92c08212584911d9 (127.6.59.129:41291): Couldn't send request to peer 5ba5c4b49af7483c92c08212584911d9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21788a4474f044e1b01424e7e6bb0687. This is attempt 1: this message will repeat every 5th retry.
I20250815 02:08:46.168531 9487 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 21788a4474f044e1b01424e7e6bb0687 with cas_config_opid_index 16: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 3)
I20250815 02:08:46.168547 9500 catalog_manager.cc:5582] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 reported cstate change: config changed from index 16 to 17, NON_VOTER 144c0179dfbb4ff499614d1b6d3cab01 (127.6.59.131) added. New cstate: current_term: 2 leader_uuid: "18b448977b9b455d93c91367f279b227" committed_config { opid_index: 17 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: NON_VOTER last_known_addr { host: "127.6.59.129" port: 41291 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: NON_VOTER last_known_addr { host: "127.6.59.131" port: 37121 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250815 02:08:46.170310 9716 consensus_peers.cc:489] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227 -> Peer 144c0179dfbb4ff499614d1b6d3cab01 (127.6.59.131:37121): Couldn't send request to peer 144c0179dfbb4ff499614d1b6d3cab01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21788a4474f044e1b01424e7e6bb0687. This is attempt 1: this message will repeat every 5th retry.
I20250815 02:08:46.186410 9761 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250815 02:08:46.186791 9903 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250815 02:08:46.187134 9603 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
e6537b4139ee49ebaaf570e251a9fb8a | 127.6.59.190:42923 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.59.148:45249 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
144c0179dfbb4ff499614d1b6d3cab01 | 127.6.59.131:37121 | HEALTHY | <none> | 1 | 0
18b448977b9b455d93c91367f279b227 | 127.6.59.130:33815 | HEALTHY | <none> | 1 | 0
5ba5c4b49af7483c92c08212584911d9 | 127.6.59.129:41291 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.6.59.129 | experimental | 127.6.59.129:41291
local_ip_for_outbound_sockets | 127.6.59.130 | experimental | 127.6.59.130:33815
local_ip_for_outbound_sockets | 127.6.59.131 | experimental | 127.6.59.131:37121
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.6.59.129:41291
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.59.130:33815
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.59.131:37121
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.59.148:45249 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
TestTable1 | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
TestTable2 | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 3
Maximum | 3
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 3
Tablets | 3
Replicas | 7
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20250815 02:08:46.195788 6382 log_verifier.cc:126] Checking tablet 21788a4474f044e1b01424e7e6bb0687
I20250815 02:08:46.203683 6382 log_verifier.cc:177] Verified matching terms for 17 ops in tablet 21788a4474f044e1b01424e7e6bb0687
I20250815 02:08:46.203743 6382 log_verifier.cc:126] Checking tablet 3ccfe0b0dfe842faa39d995d7b356dea
I20250815 02:08:46.210613 6382 log_verifier.cc:177] Verified matching terms for 12 ops in tablet 3ccfe0b0dfe842faa39d995d7b356dea
I20250815 02:08:46.210672 6382 log_verifier.cc:126] Checking tablet a38410e3c42a4394849a2a575857fc2e
I20250815 02:08:46.217384 6382 log_verifier.cc:177] Verified matching terms for 15 ops in tablet a38410e3c42a4394849a2a575857fc2e
I20250815 02:08:46.217494 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 9470
I20250815 02:08:46.223237 6382 minidump.cc:252] Setting minidump size limit to 20M
I20250815 02:08:46.223498 6382 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:46.223647 6382 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:46.225786 10006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:46.225817 10008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:46.225782 10005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:46.225839 6382 server_base.cc:1047] running on GCE node
I20250815 02:08:46.226171 6382 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250815 02:08:46.226213 6382 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250815 02:08:46.226233 6382 hybrid_clock.cc:648] HybridClock initialized: now 1755223726226234 us; error 0 us; skew 500 ppm
I20250815 02:08:46.226915 6382 webserver.cc:480] Webserver started at http://0.0.0.0:40767/ using document root <none> and password file <none>
I20250815 02:08:46.227072 6382 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:46.227104 6382 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:46.227821 6382 fs_manager.cc:714] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:46.228323 10013 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:46.228461 6382 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:46.228535 6382 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
uuid: "e6537b4139ee49ebaaf570e251a9fb8a"
format_stamp: "Formatted at 2025-08-15 02:08:44 on dist-test-slave-wq2q"
I20250815 02:08:46.228765 6382 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:46.235114 6382 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:46.235309 6382 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:46.235416 6382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:46.236210 6382 sys_catalog.cc:263] Verifying existing consensus state
W20250815 02:08:46.236538 6382 sys_catalog.cc:243] For a single master config, on-disk Raft master: 127.6.59.190:42923 exists but no master address supplied!
I20250815 02:08:46.237047 6382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Bootstrap starting.
I20250815 02:08:46.244259 6382 log.cc:826] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:46.246829 6382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=26 overwritten=0 applied=26 ignored=0} inserts{seen=13 ignored=0} mutations{seen=17 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:46.247109 6382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a: Bootstrap complete.
I20250815 02:08:46.248821 6382 raft_consensus.cc:357] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:46.248932 6382 raft_consensus.cc:738] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6537b4139ee49ebaaf570e251a9fb8a, State: Initialized, Role: FOLLOWER
I20250815 02:08:46.249032 6382 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 26, Last appended: 3.26, Last appended by leader: 26, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:46.249083 6382 raft_consensus.cc:397] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:46.249132 6382 raft_consensus.cc:491] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:46.249182 6382 raft_consensus.cc:3058] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 3 FOLLOWER]: Advancing to term 4
I20250815 02:08:46.250021 6382 raft_consensus.cc:513] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:46.250121 6382 leader_election.cc:304] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6537b4139ee49ebaaf570e251a9fb8a; no voters:
I20250815 02:08:46.250283 6382 leader_election.cc:290] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [CANDIDATE]: Term 4 election: Requested vote from peers
I20250815 02:08:46.250371 10020 raft_consensus.cc:2802] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 4 FOLLOWER]: Leader election won for term 4
I20250815 02:08:46.250505 10020 raft_consensus.cc:695] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 4 LEADER]: Becoming Leader. State: Replica: e6537b4139ee49ebaaf570e251a9fb8a, State: Running, Role: LEADER
I20250815 02:08:46.250591 10020 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 3.26, Last appended by leader: 26, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } }
I20250815 02:08:46.250845 10022 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6537b4139ee49ebaaf570e251a9fb8a. Latest consensus state: current_term: 4 leader_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } } }
I20250815 02:08:46.250914 10022 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:46.250892 10021 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 leader_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6537b4139ee49ebaaf570e251a9fb8a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42923 } } }
I20250815 02:08:46.250958 10021 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:46.252516 6382 tablet_replica.cc:331] stopping tablet replica
I20250815 02:08:46.252580 6382 raft_consensus.cc:2241] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 4 LEADER]: Raft consensus shutting down.
I20250815 02:08:46.252642 6382 raft_consensus.cc:2270] T 00000000000000000000000000000000 P e6537b4139ee49ebaaf570e251a9fb8a [term 4 FOLLOWER]: Raft consensus is shut down!
I20250815 02:08:46.252943 6382 master.cc:561] Master@0.0.0.0:7051 shutting down...
W20250815 02:08:46.253034 6382 acceptor_pool.cc:194] Could not shut down acceptor socket on 0.0.0.0:7051: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20250815 02:08:46.263918 6382 master.cc:583] Master@0.0.0.0:7051 shutdown complete.
I20250815 02:08:46.559782 10070 ts_tablet_manager.cc:927] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01: Initiating tablet copy from peer 18b448977b9b455d93c91367f279b227 (127.6.59.130:33815)
I20250815 02:08:46.560015 10070 tablet_copy_client.cc:323] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01: tablet copy: Beginning tablet copy session from remote peer at address 127.6.59.130:33815
I20250815 02:08:46.562968 9801 tablet_copy_service.cc:140] P 18b448977b9b455d93c91367f279b227: Received BeginTabletCopySession request for tablet 21788a4474f044e1b01424e7e6bb0687 from peer 144c0179dfbb4ff499614d1b6d3cab01 ({username='slave'} at 127.6.59.131:36393)
I20250815 02:08:46.563053 9801 tablet_copy_service.cc:161] P 18b448977b9b455d93c91367f279b227: Beginning new tablet copy session on tablet 21788a4474f044e1b01424e7e6bb0687 from peer 144c0179dfbb4ff499614d1b6d3cab01 at {username='slave'} at 127.6.59.131:36393: session id = 144c0179dfbb4ff499614d1b6d3cab01-21788a4474f044e1b01424e7e6bb0687
I20250815 02:08:46.563530 9801 tablet_copy_source_session.cc:215] T 21788a4474f044e1b01424e7e6bb0687 P 18b448977b9b455d93c91367f279b227: Tablet Copy: opened 0 blocks and 1 log segments
I20250815 02:08:46.564131 10070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21788a4474f044e1b01424e7e6bb0687. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:46.565357 10070 tablet_copy_client.cc:806] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01: tablet copy: Starting download of 0 data blocks...
I20250815 02:08:46.565482 10070 tablet_copy_client.cc:670] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01: tablet copy: Starting download of 1 WAL segments...
I20250815 02:08:46.566378 10070 tablet_copy_client.cc:538] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250815 02:08:46.566932 10070 tablet_bootstrap.cc:492] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01: Bootstrap starting.
I20250815 02:08:46.570202 10070 tablet_bootstrap.cc:492] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=650 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:46.570377 10070 tablet_bootstrap.cc:492] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01: Bootstrap complete.
I20250815 02:08:46.570473 10070 ts_tablet_manager.cc:1397] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20250815 02:08:46.570647 10070 raft_consensus.cc:357] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 17 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } } peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: NON_VOTER last_known_addr { host: "127.6.59.129" port: 41291 } attrs { promote: true } } peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: NON_VOTER last_known_addr { host: "127.6.59.131" port: 37121 } attrs { promote: true } }
I20250815 02:08:46.570706 10070 raft_consensus.cc:738] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 144c0179dfbb4ff499614d1b6d3cab01, State: Initialized, Role: LEARNER
I20250815 02:08:46.570746 10070 consensus_queue.cc:260] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 2.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: true peers { permanent_uuid: "18b448977b9b455d93c91367f279b227" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 33815 } } peers { permanent_uuid: "5ba5c4b49af7483c92c08212584911d9" member_type: NON_VOTER last_known_addr { host: "127.6.59.129" port: 41291 } attrs { promote: true } } peers { permanent_uuid: "144c0179dfbb4ff499614d1b6d3cab01" member_type: NON_VOTER last_known_addr { host: "127.6.59.131" port: 37121 } attrs { promote: true } }
I20250815 02:08:46.570884 10070 ts_tablet_manager.cc:1428] T 21788a4474f044e1b01424e7e6bb0687 P 144c0179dfbb4ff499614d1b6d3cab01: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:46.571251 9801 tablet_copy_service.cc:342] P 18b448977b9b455d93c91367f279b227: Request end of tablet copy session 144c0179dfbb4ff499614d1b6d3cab01-21788a4474f044e1b01424e7e6bb0687 received from {username='slave'} at 127.6.59.131:36393
I20250815 02:08:46.571324 9801 tablet_copy_service.cc:434] P 18b448977b9b455d93c91367f279b227: ending tablet copy session 144c0179dfbb4ff499614d1b6d3cab01-21788a4474f044e1b01424e7e6bb0687 on tablet 21788a4474f044e1b01424e7e6bb0687 with peer 144c0179dfbb4ff499614d1b6d3cab01
W20250815 02:08:46.571553 9969 heartbeater.cc:646] Failed to heartbeat to 127.6.59.190:42923 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.59.190:42923: connect: Connection refused (error 111)
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tools/kudu-admin-test.cc:3908: Failure
Value of: stdout2
Expected: has substring "Rebuilt from 3 replicas, of which 0 had errors"
Actual: "Rebuild Report\nTablet Servers\n address | status\n--------------------+--------\n 127.6.59.129:41291 | OK\n 127.6.59.130:33815 | OK\n 127.6.59.131:37121 | OK\nRebuilt from 3 tablet servers, of which 0 had errors\n\nReplicas\n table | tablet | tablet server | status\n-----------+----------------------------------+--------------------+--------\n TestTable | 3ccfe0b0dfe842faa39d995d7b356dea | 127.6.59.129:41291 | OK\nRebuilt from 1 replicas, of which 0 had errors\n"
I20250815 02:08:46.577232 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 9558
I20250815 02:08:46.582075 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 9692
I20250815 02:08:46.586524 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 9831
2025-08-15T02:08:46Z chronyd exiting
I20250815 02:08:46.602336 6382 test_util.cc:183] -----------------------------------------------
I20250815 02:08:46.602409 6382 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1755223715463109-6382-0
[ FAILED ] AdminCliTest.TestRebuildTables (1879 ms)
[----------] 5 tests from AdminCliTest (11135 ms total)
[----------] 1 test from EnableKudu1097AndDownTS/MoveTabletParamTest
[ RUN ] EnableKudu1097AndDownTS/MoveTabletParamTest.Test/4
I20250815 02:08:46.602860 6382 test_util.cc:276] Using random seed: -558253463
I20250815 02:08:46.603386 6382 ts_itest-base.cc:115] Starting cluster with:
I20250815 02:08:46.603423 6382 ts_itest-base.cc:116] --------------
I20250815 02:08:46.603444 6382 ts_itest-base.cc:117] 5 tablet servers
I20250815 02:08:46.603462 6382 ts_itest-base.cc:118] 3 replicas per TS
I20250815 02:08:46.603488 6382 ts_itest-base.cc:119] --------------
2025-08-15T02:08:46Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-15T02:08:46Z Disabled control of system clock
I20250815 02:08:46.611023 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:43741
--webserver_interface=127.6.59.190
--webserver_port=0
--builtin_ntp_servers=127.6.59.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:43741
--raft_prepare_replacement_before_eviction=true with env {}
W20250815 02:08:46.687628 10080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:46.687819 10080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:46.687852 10080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:46.689306 10080 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250815 02:08:46.689349 10080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:46.689371 10080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:46.689392 10080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:46.689411 10080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:46.690910 10080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:43741
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:43741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.10080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:46.691164 10080 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:46.691382 10080 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:46.693701 10086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:46.693687 10085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:46.693876 10080 server_base.cc:1047] running on GCE node
W20250815 02:08:46.693688 10088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:46.694176 10080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:46.694414 10080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:46.695540 10080 hybrid_clock.cc:648] HybridClock initialized: now 1755223726695521 us; error 36 us; skew 500 ppm
I20250815 02:08:46.696672 10080 webserver.cc:480] Webserver started at http://127.6.59.190:33117/ using document root <none> and password file <none>
I20250815 02:08:46.696884 10080 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:46.696924 10080 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:46.697048 10080 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:46.697896 10080 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "0a666495722249d29c6fe6134ed50168"
format_stamp: "Formatted at 2025-08-15 02:08:46 on dist-test-slave-wq2q"
I20250815 02:08:46.698251 10080 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "0a666495722249d29c6fe6134ed50168"
format_stamp: "Formatted at 2025-08-15 02:08:46 on dist-test-slave-wq2q"
I20250815 02:08:46.699395 10080 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:46.700104 10094 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:46.700294 10080 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:46.700368 10080 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
uuid: "0a666495722249d29c6fe6134ed50168"
format_stamp: "Formatted at 2025-08-15 02:08:46 on dist-test-slave-wq2q"
I20250815 02:08:46.700418 10080 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:46.713788 10080 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:46.714174 10080 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:46.714293 10080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:46.717949 10080 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:43741
I20250815 02:08:46.718052 10146 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:43741 every 8 connection(s)
I20250815 02:08:46.718344 10080 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250815 02:08:46.718935 10147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:46.721071 10147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168: Bootstrap starting.
I20250815 02:08:46.721670 10147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:46.721923 10147 log.cc:826] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:46.722522 10147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168: No bootstrap required, opened a new log
I20250815 02:08:46.723601 10147 raft_consensus.cc:357] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a666495722249d29c6fe6134ed50168" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43741 } }
I20250815 02:08:46.723714 10147 raft_consensus.cc:383] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:46.723747 10147 raft_consensus.cc:738] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a666495722249d29c6fe6134ed50168, State: Initialized, Role: FOLLOWER
I20250815 02:08:46.723898 10147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a666495722249d29c6fe6134ed50168" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43741 } }
I20250815 02:08:46.723975 10147 raft_consensus.cc:397] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:46.724049 10147 raft_consensus.cc:491] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:46.724098 10147 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:46.724601 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 10080
I20250815 02:08:46.724725 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250815 02:08:46.724713 10147 raft_consensus.cc:513] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a666495722249d29c6fe6134ed50168" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43741 } }
I20250815 02:08:46.724814 10147 leader_election.cc:304] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a666495722249d29c6fe6134ed50168; no voters:
I20250815 02:08:46.724979 10147 leader_election.cc:290] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:46.725030 10150 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:46.725195 10150 raft_consensus.cc:695] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [term 1 LEADER]: Becoming Leader. State: Replica: 0a666495722249d29c6fe6134ed50168, State: Running, Role: LEADER
I20250815 02:08:46.725219 10147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:46.725342 10150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a666495722249d29c6fe6134ed50168" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43741 } }
I20250815 02:08:46.725632 10151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a666495722249d29c6fe6134ed50168" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a666495722249d29c6fe6134ed50168" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43741 } } }
I20250815 02:08:46.725723 10151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:46.725648 10150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a666495722249d29c6fe6134ed50168. Latest consensus state: current_term: 1 leader_uuid: "0a666495722249d29c6fe6134ed50168" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a666495722249d29c6fe6134ed50168" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43741 } } }
I20250815 02:08:46.725842 10150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168 [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:46.726013 10156 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:46.726712 10156 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:46.728229 10156 catalog_manager.cc:1349] Generated new cluster ID: a9f9eeaa88f8419d8349876c59a3ffdd
I20250815 02:08:46.728279 10156 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:46.736958 10156 catalog_manager.cc:1372] Generated new certificate authority record
I20250815 02:08:46.737686 10156 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:46.751322 10156 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 0a666495722249d29c6fe6134ed50168: Generated new TSK 0
I20250815 02:08:46.751533 10156 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250815 02:08:46.755086 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.129:0
--local_ip_for_outbound_sockets=127.6.59.129
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43741
--builtin_ntp_servers=127.6.59.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20250815 02:08:46.832935 10171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:46.833120 10171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:46.833153 10171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:46.834705 10171 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250815 02:08:46.834749 10171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:46.834789 10171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.129
I20250815 02:08:46.836241 10171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43741
--never_fsync=true
--heap_profile_path=/tmp/kudu.10171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.129
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:46.836460 10171 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:46.836680 10171 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:46.839103 10177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:46.839129 10176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:46.839257 10171 server_base.cc:1047] running on GCE node
W20250815 02:08:46.839382 10179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:46.839570 10171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:46.839778 10171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:46.840915 10171 hybrid_clock.cc:648] HybridClock initialized: now 1755223726840906 us; error 34 us; skew 500 ppm
I20250815 02:08:46.842037 10171 webserver.cc:480] Webserver started at http://127.6.59.129:44821/ using document root <none> and password file <none>
I20250815 02:08:46.842234 10171 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:46.842273 10171 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:46.842375 10171 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:46.843506 10171 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "4fbfd85d9b1a4b3186706e72249cbab3"
format_stamp: "Formatted at 2025-08-15 02:08:46 on dist-test-slave-wq2q"
I20250815 02:08:46.844913 10171 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "4fbfd85d9b1a4b3186706e72249cbab3"
format_stamp: "Formatted at 2025-08-15 02:08:46 on dist-test-slave-wq2q"
I20250815 02:08:46.846184 10171 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250815 02:08:46.846884 10185 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:46.847066 10171 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:46.847126 10171 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "4fbfd85d9b1a4b3186706e72249cbab3"
format_stamp: "Formatted at 2025-08-15 02:08:46 on dist-test-slave-wq2q"
I20250815 02:08:46.847188 10171 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:46.862763 10171 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:46.863014 10171 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:46.863121 10171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:46.863368 10171 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:46.863710 10171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:46.863741 10171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:46.863765 10171 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:46.863780 10171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:46.870877 10171 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.129:43515
I20250815 02:08:46.870940 10298 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.129:43515 every 8 connection(s)
I20250815 02:08:46.871270 10171 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250815 02:08:46.875967 10299 heartbeater.cc:344] Connected to a master server at 127.6.59.190:43741
I20250815 02:08:46.876080 10299 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:46.876327 10299 heartbeater.cc:507] Master 127.6.59.190:43741 requested a full tablet report, sending...
I20250815 02:08:46.876778 10111 ts_manager.cc:194] Registered new tserver with Master: 4fbfd85d9b1a4b3186706e72249cbab3 (127.6.59.129:43515)
I20250815 02:08:46.877378 10111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.129:58995
I20250815 02:08:46.880110 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 10171
I20250815 02:08:46.880201 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250815 02:08:46.881134 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.130:0
--local_ip_for_outbound_sockets=127.6.59.130
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43741
--builtin_ntp_servers=127.6.59.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20250815 02:08:46.956840 10302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:46.957000 10302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:46.957019 10302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:46.958405 10302 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250815 02:08:46.958448 10302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:46.958472 10302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.130
I20250815 02:08:46.959893 10302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43741
--never_fsync=true
--heap_profile_path=/tmp/kudu.10302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.130
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:46.960105 10302 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:46.960314 10302 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:46.962728 10310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:46.962882 10302 server_base.cc:1047] running on GCE node
W20250815 02:08:46.962754 10308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:46.962745 10307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:46.963104 10302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:46.963268 10302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:46.964388 10302 hybrid_clock.cc:648] HybridClock initialized: now 1755223726964377 us; error 27 us; skew 500 ppm
I20250815 02:08:46.965500 10302 webserver.cc:480] Webserver started at http://127.6.59.130:34353/ using document root <none> and password file <none>
I20250815 02:08:46.965698 10302 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:46.965734 10302 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:46.965842 10302 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:46.966687 10302 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "41ebfb8a38b74b7789700d107a8a2aa9"
format_stamp: "Formatted at 2025-08-15 02:08:46 on dist-test-slave-wq2q"
I20250815 02:08:46.966970 10302 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "41ebfb8a38b74b7789700d107a8a2aa9"
format_stamp: "Formatted at 2025-08-15 02:08:46 on dist-test-slave-wq2q"
I20250815 02:08:46.968058 10302 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250815 02:08:46.968704 10316 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:46.968853 10302 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:46.968923 10302 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "41ebfb8a38b74b7789700d107a8a2aa9"
format_stamp: "Formatted at 2025-08-15 02:08:46 on dist-test-slave-wq2q"
I20250815 02:08:46.968979 10302 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:46.983631 10302 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:46.983880 10302 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:46.983992 10302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:46.984215 10302 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:46.984522 10302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:46.984560 10302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:46.984593 10302 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:46.984629 10302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:46.990255 10302 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.130:45567
I20250815 02:08:46.990358 10429 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.130:45567 every 8 connection(s)
I20250815 02:08:46.990644 10302 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250815 02:08:46.995039 10430 heartbeater.cc:344] Connected to a master server at 127.6.59.190:43741
I20250815 02:08:46.995123 10430 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:46.995188 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 10302
I20250815 02:08:46.995296 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250815 02:08:46.995339 10430 heartbeater.cc:507] Master 127.6.59.190:43741 requested a full tablet report, sending...
I20250815 02:08:46.995713 10111 ts_manager.cc:194] Registered new tserver with Master: 41ebfb8a38b74b7789700d107a8a2aa9 (127.6.59.130:45567)
I20250815 02:08:46.996078 10111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.130:38031
I20250815 02:08:46.996464 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.131:0
--local_ip_for_outbound_sockets=127.6.59.131
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43741
--builtin_ntp_servers=127.6.59.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20250815 02:08:47.074759 10433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:47.074916 10433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:47.074935 10433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:47.076351 10433 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250815 02:08:47.076393 10433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:47.076417 10433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.131
I20250815 02:08:47.077858 10433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43741
--never_fsync=true
--heap_profile_path=/tmp/kudu.10433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.131
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:47.078087 10433 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:47.078299 10433 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:47.080670 10439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:47.080757 10433 server_base.cc:1047] running on GCE node
W20250815 02:08:47.080686 10438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:47.080688 10441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:47.081027 10433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:47.081239 10433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:47.082368 10433 hybrid_clock.cc:648] HybridClock initialized: now 1755223727082357 us; error 30 us; skew 500 ppm
I20250815 02:08:47.083423 10433 webserver.cc:480] Webserver started at http://127.6.59.131:41973/ using document root <none> and password file <none>
I20250815 02:08:47.083626 10433 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:47.083664 10433 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:47.083770 10433 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:47.084611 10433 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "cf8c6eed75e34db4922156788b388d63"
format_stamp: "Formatted at 2025-08-15 02:08:47 on dist-test-slave-wq2q"
I20250815 02:08:47.084899 10433 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "cf8c6eed75e34db4922156788b388d63"
format_stamp: "Formatted at 2025-08-15 02:08:47 on dist-test-slave-wq2q"
I20250815 02:08:47.085973 10433 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:47.086727 10447 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:47.086884 10433 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:47.086953 10433 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "cf8c6eed75e34db4922156788b388d63"
format_stamp: "Formatted at 2025-08-15 02:08:47 on dist-test-slave-wq2q"
I20250815 02:08:47.087011 10433 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:47.095520 10433 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:47.095757 10433 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:47.095875 10433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:47.096055 10433 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:47.096323 10433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:47.096352 10433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:47.096387 10433 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:47.096417 10433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:47.101747 10433 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.131:33611
I20250815 02:08:47.101825 10560 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.131:33611 every 8 connection(s)
I20250815 02:08:47.102119 10433 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250815 02:08:47.106339 10561 heartbeater.cc:344] Connected to a master server at 127.6.59.190:43741
I20250815 02:08:47.106429 10561 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:47.106617 10561 heartbeater.cc:507] Master 127.6.59.190:43741 requested a full tablet report, sending...
I20250815 02:08:47.106909 10111 ts_manager.cc:194] Registered new tserver with Master: cf8c6eed75e34db4922156788b388d63 (127.6.59.131:33611)
I20250815 02:08:47.107277 10111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.131:45587
I20250815 02:08:47.110699 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 10433
I20250815 02:08:47.110805 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250815 02:08:47.111756 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.132:0
--local_ip_for_outbound_sockets=127.6.59.132
--webserver_interface=127.6.59.132
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43741
--builtin_ntp_servers=127.6.59.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20250815 02:08:47.189929 10564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:47.190097 10564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:47.190114 10564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:47.191496 10564 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250815 02:08:47.191540 10564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:47.191562 10564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.132
I20250815 02:08:47.192981 10564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.6.59.132
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43741
--never_fsync=true
--heap_profile_path=/tmp/kudu.10564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.132
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:47.193183 10564 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:47.193394 10564 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:47.195760 10570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:47.195878 10564 server_base.cc:1047] running on GCE node
W20250815 02:08:47.195751 10572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:47.195777 10569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:47.196132 10564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:47.196306 10564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:47.197431 10564 hybrid_clock.cc:648] HybridClock initialized: now 1755223727197418 us; error 26 us; skew 500 ppm
I20250815 02:08:47.198472 10564 webserver.cc:480] Webserver started at http://127.6.59.132:45035/ using document root <none> and password file <none>
I20250815 02:08:47.198668 10564 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:47.198709 10564 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:47.198817 10564 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:47.199640 10564 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "97e6a27c53e04e1f9ac755ed3913a392"
format_stamp: "Formatted at 2025-08-15 02:08:47 on dist-test-slave-wq2q"
I20250815 02:08:47.199926 10564 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "97e6a27c53e04e1f9ac755ed3913a392"
format_stamp: "Formatted at 2025-08-15 02:08:47 on dist-test-slave-wq2q"
I20250815 02:08:47.201017 10564 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20250815 02:08:47.201640 10578 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:47.201794 10564 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:47.201861 10564 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "97e6a27c53e04e1f9ac755ed3913a392"
format_stamp: "Formatted at 2025-08-15 02:08:47 on dist-test-slave-wq2q"
I20250815 02:08:47.201920 10564 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:47.216230 10564 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:47.216463 10564 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:47.216573 10564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:47.216780 10564 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:47.217096 10564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:47.217128 10564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:47.217164 10564 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:47.217190 10564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:47.223016 10564 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.132:38231
I20250815 02:08:47.223075 10691 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.132:38231 every 8 connection(s)
I20250815 02:08:47.223341 10564 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20250815 02:08:47.226604 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 10564
I20250815 02:08:47.226696 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20250815 02:08:47.227478 10692 heartbeater.cc:344] Connected to a master server at 127.6.59.190:43741
I20250815 02:08:47.227872 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.133:0
--local_ip_for_outbound_sockets=127.6.59.133
--webserver_interface=127.6.59.133
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43741
--builtin_ntp_servers=127.6.59.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20250815 02:08:47.227599 10692 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:47.229282 10692 heartbeater.cc:507] Master 127.6.59.190:43741 requested a full tablet report, sending...
I20250815 02:08:47.229643 10111 ts_manager.cc:194] Registered new tserver with Master: 97e6a27c53e04e1f9ac755ed3913a392 (127.6.59.132:38231)
I20250815 02:08:47.230027 10111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.132:33401
W20250815 02:08:47.307541 10695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:47.307739 10695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:47.307771 10695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:47.309140 10695 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250815 02:08:47.309185 10695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:47.309219 10695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.133
I20250815 02:08:47.310652 10695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.6.59.133
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43741
--never_fsync=true
--heap_profile_path=/tmp/kudu.10695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.133
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:47.310864 10695 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:47.311079 10695 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:47.313467 10703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:47.313467 10701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:47.313720 10700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:47.314487 10695 server_base.cc:1047] running on GCE node
I20250815 02:08:47.314654 10695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:47.314863 10695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:47.315996 10695 hybrid_clock.cc:648] HybridClock initialized: now 1755223727315975 us; error 32 us; skew 500 ppm
I20250815 02:08:47.317020 10695 webserver.cc:480] Webserver started at http://127.6.59.133:40095/ using document root <none> and password file <none>
I20250815 02:08:47.317204 10695 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:47.317248 10695 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:47.317368 10695 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:47.318238 10695 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "9f1e2d72160141ae86c2970dbc15005e"
format_stamp: "Formatted at 2025-08-15 02:08:47 on dist-test-slave-wq2q"
I20250815 02:08:47.318543 10695 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "9f1e2d72160141ae86c2970dbc15005e"
format_stamp: "Formatted at 2025-08-15 02:08:47 on dist-test-slave-wq2q"
I20250815 02:08:47.319701 10695 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:47.320338 10709 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:47.320483 10695 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:47.320551 10695 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "9f1e2d72160141ae86c2970dbc15005e"
format_stamp: "Formatted at 2025-08-15 02:08:47 on dist-test-slave-wq2q"
I20250815 02:08:47.320611 10695 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:47.353008 10695 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:47.353281 10695 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:47.353395 10695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:47.353638 10695 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:47.353952 10695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:47.354004 10695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:47.354044 10695 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:47.354075 10695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:47.360235 10695 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.133:40909
I20250815 02:08:47.360293 10822 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.133:40909 every 8 connection(s)
I20250815 02:08:47.360580 10695 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20250815 02:08:47.362562 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 10695
I20250815 02:08:47.362677 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20250815 02:08:47.365051 10823 heartbeater.cc:344] Connected to a master server at 127.6.59.190:43741
I20250815 02:08:47.365145 10823 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:47.365339 10823 heartbeater.cc:507] Master 127.6.59.190:43741 requested a full tablet report, sending...
I20250815 02:08:47.365633 10111 ts_manager.cc:194] Registered new tserver with Master: 9f1e2d72160141ae86c2970dbc15005e (127.6.59.133:40909)
I20250815 02:08:47.366037 10111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.133:38891
I20250815 02:08:47.374651 6382 external_mini_cluster.cc:949] 5 TS(s) registered with all masters
I20250815 02:08:47.380007 10111 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50508:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20250815 02:08:47.387257 10233 tablet_service.cc:1468] Processing CreateTablet for tablet 67e5fa347aa940ddb7bcb990591b24e4 (DEFAULT_TABLE table=TestTable [id=8e3ceda30e2a49909ca898d9dcc0190d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:47.387279 10495 tablet_service.cc:1468] Processing CreateTablet for tablet 67e5fa347aa940ddb7bcb990591b24e4 (DEFAULT_TABLE table=TestTable [id=8e3ceda30e2a49909ca898d9dcc0190d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:47.387532 10495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e5fa347aa940ddb7bcb990591b24e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:47.387535 10233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e5fa347aa940ddb7bcb990591b24e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:47.389616 10840 tablet_bootstrap.cc:492] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63: Bootstrap starting.
I20250815 02:08:47.389854 10757 tablet_service.cc:1468] Processing CreateTablet for tablet 67e5fa347aa940ddb7bcb990591b24e4 (DEFAULT_TABLE table=TestTable [id=8e3ceda30e2a49909ca898d9dcc0190d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:47.389968 10841 tablet_bootstrap.cc:492] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3: Bootstrap starting.
I20250815 02:08:47.390158 10757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e5fa347aa940ddb7bcb990591b24e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:47.390731 10840 tablet_bootstrap.cc:654] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:47.390913 10841 tablet_bootstrap.cc:654] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:47.390976 10840 log.cc:826] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:47.391286 10841 log.cc:826] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:47.391664 10840 tablet_bootstrap.cc:492] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63: No bootstrap required, opened a new log
I20250815 02:08:47.391729 10840 ts_tablet_manager.cc:1397] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250815 02:08:47.391894 10841 tablet_bootstrap.cc:492] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3: No bootstrap required, opened a new log
I20250815 02:08:47.391956 10841 ts_tablet_manager.cc:1397] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250815 02:08:47.392728 10844 tablet_bootstrap.cc:492] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e: Bootstrap starting.
I20250815 02:08:47.392860 10840 raft_consensus.cc:357] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:47.393079 10840 raft_consensus.cc:383] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:47.393118 10840 raft_consensus.cc:738] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf8c6eed75e34db4922156788b388d63, State: Initialized, Role: FOLLOWER
I20250815 02:08:47.393241 10840 consensus_queue.cc:260] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:47.393350 10841 raft_consensus.cc:357] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:47.393474 10841 raft_consensus.cc:383] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:47.393472 10840 ts_tablet_manager.cc:1428] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250815 02:08:47.393512 10841 raft_consensus.cc:738] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fbfd85d9b1a4b3186706e72249cbab3, State: Initialized, Role: FOLLOWER
I20250815 02:08:47.393580 10844 tablet_bootstrap.cc:654] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:47.393587 10561 heartbeater.cc:499] Master 127.6.59.190:43741 was elected leader, sending a full tablet report...
I20250815 02:08:47.393597 10841 consensus_queue.cc:260] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:47.393844 10841 ts_tablet_manager.cc:1428] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250815 02:08:47.393890 10299 heartbeater.cc:499] Master 127.6.59.190:43741 was elected leader, sending a full tablet report...
I20250815 02:08:47.393925 10844 log.cc:826] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:47.394640 10844 tablet_bootstrap.cc:492] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e: No bootstrap required, opened a new log
I20250815 02:08:47.394701 10844 ts_tablet_manager.cc:1397] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250815 02:08:47.395735 10844 raft_consensus.cc:357] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:47.395830 10844 raft_consensus.cc:383] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:47.395854 10844 raft_consensus.cc:738] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e2d72160141ae86c2970dbc15005e, State: Initialized, Role: FOLLOWER
I20250815 02:08:47.395921 10844 consensus_queue.cc:260] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:47.396121 10844 ts_tablet_manager.cc:1428] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20250815 02:08:47.396159 10823 heartbeater.cc:499] Master 127.6.59.190:43741 was elected leader, sending a full tablet report...
I20250815 02:08:47.416464 10845 raft_consensus.cc:491] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250815 02:08:47.416587 10845 raft_consensus.cc:513] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:47.416916 10845 leader_election.cc:290] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fbfd85d9b1a4b3186706e72249cbab3 (127.6.59.129:43515), 9f1e2d72160141ae86c2970dbc15005e (127.6.59.133:40909)
I20250815 02:08:47.419593 10777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "67e5fa347aa940ddb7bcb990591b24e4" candidate_uuid: "cf8c6eed75e34db4922156788b388d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e2d72160141ae86c2970dbc15005e" is_pre_election: true
I20250815 02:08:47.419749 10777 raft_consensus.cc:2466] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf8c6eed75e34db4922156788b388d63 in term 0.
I20250815 02:08:47.419961 10451 leader_election.cc:304] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e2d72160141ae86c2970dbc15005e, cf8c6eed75e34db4922156788b388d63; no voters:
I20250815 02:08:47.420086 10845 raft_consensus.cc:2802] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250815 02:08:47.420142 10845 raft_consensus.cc:491] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250815 02:08:47.420158 10845 raft_consensus.cc:3058] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:47.420099 10253 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "67e5fa347aa940ddb7bcb990591b24e4" candidate_uuid: "cf8c6eed75e34db4922156788b388d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" is_pre_election: true
I20250815 02:08:47.420351 10253 raft_consensus.cc:2466] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf8c6eed75e34db4922156788b388d63 in term 0.
I20250815 02:08:47.420794 10845 raft_consensus.cc:513] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:47.420907 10845 leader_election.cc:290] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [CANDIDATE]: Term 1 election: Requested vote from peers 4fbfd85d9b1a4b3186706e72249cbab3 (127.6.59.129:43515), 9f1e2d72160141ae86c2970dbc15005e (127.6.59.133:40909)
I20250815 02:08:47.421056 10777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "67e5fa347aa940ddb7bcb990591b24e4" candidate_uuid: "cf8c6eed75e34db4922156788b388d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e2d72160141ae86c2970dbc15005e"
I20250815 02:08:47.421120 10777 raft_consensus.cc:3058] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:47.422086 10253 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "67e5fa347aa940ddb7bcb990591b24e4" candidate_uuid: "cf8c6eed75e34db4922156788b388d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fbfd85d9b1a4b3186706e72249cbab3"
I20250815 02:08:47.422152 10253 raft_consensus.cc:3058] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:47.422323 10777 raft_consensus.cc:2466] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf8c6eed75e34db4922156788b388d63 in term 1.
I20250815 02:08:47.422492 10451 leader_election.cc:304] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e2d72160141ae86c2970dbc15005e, cf8c6eed75e34db4922156788b388d63; no voters:
I20250815 02:08:47.422583 10845 raft_consensus.cc:2802] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:47.422761 10845 raft_consensus.cc:695] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [term 1 LEADER]: Becoming Leader. State: Replica: cf8c6eed75e34db4922156788b388d63, State: Running, Role: LEADER
I20250815 02:08:47.422876 10253 raft_consensus.cc:2466] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf8c6eed75e34db4922156788b388d63 in term 1.
I20250815 02:08:47.422875 10845 consensus_queue.cc:237] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:47.423622 10110 catalog_manager.cc:5582] T 67e5fa347aa940ddb7bcb990591b24e4 P cf8c6eed75e34db4922156788b388d63 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf8c6eed75e34db4922156788b388d63 (127.6.59.131). New cstate: current_term: 1 leader_uuid: "cf8c6eed75e34db4922156788b388d63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } health_report { overall_health: UNKNOWN } } }
I20250815 02:08:47.429296 6382 external_mini_cluster.cc:949] 5 TS(s) registered with all masters
I20250815 02:08:47.429692 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 4fbfd85d9b1a4b3186706e72249cbab3 to finish bootstrapping
I20250815 02:08:47.432826 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver cf8c6eed75e34db4922156788b388d63 to finish bootstrapping
I20250815 02:08:47.435544 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 9f1e2d72160141ae86c2970dbc15005e to finish bootstrapping
I20250815 02:08:47.438517 6382 test_util.cc:276] Using random seed: -557417804
I20250815 02:08:47.443758 6382 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20250815 02:08:47.443910 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 10433
W20250815 02:08:47.449839 10857 negotiation.cc:337] Failed RPC negotiation. Trace:
0815 02:08:47.447796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.59.131:33611 (local address 127.0.0.1:56716)
0815 02:08:47.447850 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0815 02:08:47.447862 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0815 02:08:47.447907 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0815 02:08:47.449675 (+ 1768us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.59.131:33611: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":32}
W20250815 02:08:47.451581 10853 meta_cache.cc:302] tablet 67e5fa347aa940ddb7bcb990591b24e4: replica cf8c6eed75e34db4922156788b388d63 (127.6.59.131:33611) has failed: Network error: Client connection negotiation failed: client connection to 127.6.59.131:33611: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250815 02:08:47.460971 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:47.466782 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:47.482506 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:47.490329 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:47.508379 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:47.521255 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:47.550139 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:47.565717 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
I20250815 02:08:47.566171 10757 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:47.567299 10626 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:47.567729 10364 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:47.568094 10233 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250815 02:08:47.592460 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:47.608260 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:47.611019 10824 tablet.cc:2378] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250815 02:08:47.622103 10300 tablet.cc:2378] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250815 02:08:47.645215 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:47.663216 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:47.702329 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:47.727128 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:47.775177 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:47.800024 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:47.858067 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:47.886000 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:47.949038 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:47.978848 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
I20250815 02:08:47.996809 10430 heartbeater.cc:499] Master 127.6.59.190:43741 was elected leader, sending a full tablet report...
W20250815 02:08:48.043838 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:48.080766 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:48.152704 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:48.191573 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
I20250815 02:08:48.230873 10692 heartbeater.cc:499] Master 127.6.59.190:43741 was elected leader, sending a full tablet report...
W20250815 02:08:48.271656 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:48.310544 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:48.394644 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:48.440557 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:48.484866 10853 meta_cache.cc:302] tablet 67e5fa347aa940ddb7bcb990591b24e4: replica cf8c6eed75e34db4922156788b388d63 (127.6.59.131:33611) has failed: Network error: Client connection negotiation failed: client connection to 127.6.59.131:33611: connect: Connection refused (error 111)
W20250815 02:08:48.529471 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:48.575433 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
I20250815 02:08:48.602869 10233 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:48.603385 10626 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:48.603443 10364 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:48.604775 10757 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250815 02:08:48.671227 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:48.723006 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:48.826015 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:48.877863 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
W20250815 02:08:48.989013 10213 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:47390: Illegal state: replica 4fbfd85d9b1a4b3186706e72249cbab3 is not leader of this config: current role FOLLOWER
W20250815 02:08:49.042994 10737 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:49420: Illegal state: replica 9f1e2d72160141ae86c2970dbc15005e is not leader of this config: current role FOLLOWER
I20250815 02:08:49.095144 10924 raft_consensus.cc:491] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250815 02:08:49.095232 10924 raft_consensus.cc:513] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:49.095537 10924 leader_election.cc:290] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cf8c6eed75e34db4922156788b388d63 (127.6.59.131:33611), 9f1e2d72160141ae86c2970dbc15005e (127.6.59.133:40909)
W20250815 02:08:49.095963 10186 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.59.131:33611: connect: Connection refused (error 111)
W20250815 02:08:49.096813 10186 leader_election.cc:336] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cf8c6eed75e34db4922156788b388d63 (127.6.59.131:33611): Network error: Client connection negotiation failed: client connection to 127.6.59.131:33611: connect: Connection refused (error 111)
I20250815 02:08:49.098595 10777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "67e5fa347aa940ddb7bcb990591b24e4" candidate_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e2d72160141ae86c2970dbc15005e" is_pre_election: true
I20250815 02:08:49.098695 10777 raft_consensus.cc:2466] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fbfd85d9b1a4b3186706e72249cbab3 in term 1.
I20250815 02:08:49.098894 10189 leader_election.cc:304] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4fbfd85d9b1a4b3186706e72249cbab3, 9f1e2d72160141ae86c2970dbc15005e; no voters: cf8c6eed75e34db4922156788b388d63
I20250815 02:08:49.099014 10924 raft_consensus.cc:2802] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250815 02:08:49.099066 10924 raft_consensus.cc:491] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250815 02:08:49.099104 10924 raft_consensus.cc:3058] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:49.099700 10924 raft_consensus.cc:513] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:49.099812 10924 leader_election.cc:290] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [CANDIDATE]: Term 2 election: Requested vote from peers cf8c6eed75e34db4922156788b388d63 (127.6.59.131:33611), 9f1e2d72160141ae86c2970dbc15005e (127.6.59.133:40909)
I20250815 02:08:49.100122 10777 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "67e5fa347aa940ddb7bcb990591b24e4" candidate_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e2d72160141ae86c2970dbc15005e"
I20250815 02:08:49.100219 10777 raft_consensus.cc:3058] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:49.100868 10777 raft_consensus.cc:2466] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fbfd85d9b1a4b3186706e72249cbab3 in term 2.
I20250815 02:08:49.101018 10189 leader_election.cc:304] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fbfd85d9b1a4b3186706e72249cbab3, 9f1e2d72160141ae86c2970dbc15005e; no voters:
W20250815 02:08:49.101130 10186 leader_election.cc:336] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cf8c6eed75e34db4922156788b388d63 (127.6.59.131:33611): Network error: Client connection negotiation failed: client connection to 127.6.59.131:33611: connect: Connection refused (error 111)
I20250815 02:08:49.101126 10924 raft_consensus.cc:2802] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 2 FOLLOWER]: Leader election won for term 2
I20250815 02:08:49.101341 10924 raft_consensus.cc:695] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [term 2 LEADER]: Becoming Leader. State: Replica: 4fbfd85d9b1a4b3186706e72249cbab3, State: Running, Role: LEADER
I20250815 02:08:49.101495 10924 consensus_queue.cc:237] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } }
I20250815 02:08:49.102299 10110 catalog_manager.cc:5582] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 reported cstate change: term changed from 1 to 2, leader changed from cf8c6eed75e34db4922156788b388d63 (127.6.59.131) to 4fbfd85d9b1a4b3186706e72249cbab3 (127.6.59.129). New cstate: current_term: 2 leader_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8c6eed75e34db4922156788b388d63" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fbfd85d9b1a4b3186706e72249cbab3" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 43515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 } health_report { overall_health: UNKNOWN } } }
I20250815 02:08:49.159406 10777 raft_consensus.cc:1273] T 67e5fa347aa940ddb7bcb990591b24e4 P 9f1e2d72160141ae86c2970dbc15005e [term 2 FOLLOWER]: Refusing update from remote peer 4fbfd85d9b1a4b3186706e72249cbab3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20250815 02:08:49.159658 10186 consensus_peers.cc:489] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 -> Peer cf8c6eed75e34db4922156788b388d63 (127.6.59.131:33611): Couldn't send request to peer cf8c6eed75e34db4922156788b388d63. Status: Network error: Client connection negotiation failed: client connection to 127.6.59.131:33611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250815 02:08:49.159691 10928 consensus_queue.cc:1035] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e2d72160141ae86c2970dbc15005e" member_type: VOTER last_known_addr { host: "127.6.59.133" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:49.642534 10364 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:49.643000 10626 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:49.647850 10757 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:49.648057 10233 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250815 02:08:50.687935 10233 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250815 02:08:50.688267 10364 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:50.688598 10626 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:50.697633 10757 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250815 02:08:51.693711 10186 consensus_peers.cc:489] T 67e5fa347aa940ddb7bcb990591b24e4 P 4fbfd85d9b1a4b3186706e72249cbab3 -> Peer cf8c6eed75e34db4922156788b388d63 (127.6.59.131:33611): Couldn't send request to peer cf8c6eed75e34db4922156788b388d63. Status: Network error: Client connection negotiation failed: client connection to 127.6.59.131:33611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250815 02:08:51.736646 10626 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:51.736956 10233 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250815 02:08:51.736972 10364 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:51.737174 10757 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:53.045116 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 10171
I20250815 02:08:53.050642 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 10302
I20250815 02:08:53.054744 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 10564
I20250815 02:08:53.058912 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 10695
I20250815 02:08:53.063304 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 10080
2025-08-15T02:08:53Z chronyd exiting
[ OK ] EnableKudu1097AndDownTS/MoveTabletParamTest.Test/4 (6480 ms)
[----------] 1 test from EnableKudu1097AndDownTS/MoveTabletParamTest (6480 ms total)
[----------] 1 test from ListTableCliSimpleParamTest
[ RUN ] ListTableCliSimpleParamTest.TestListTables/2
I20250815 02:08:53.083779 6382 test_util.cc:276] Using random seed: -551772545
I20250815 02:08:53.084281 6382 ts_itest-base.cc:115] Starting cluster with:
I20250815 02:08:53.084322 6382 ts_itest-base.cc:116] --------------
I20250815 02:08:53.084333 6382 ts_itest-base.cc:117] 1 tablet servers
I20250815 02:08:53.084348 6382 ts_itest-base.cc:118] 1 replicas per TS
I20250815 02:08:53.084372 6382 ts_itest-base.cc:119] --------------
2025-08-15T02:08:53Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-15T02:08:53Z Disabled control of system clock
I20250815 02:08:53.091811 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:43049
--webserver_interface=127.6.59.190
--webserver_port=0
--builtin_ntp_servers=127.6.59.148:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:43049 with env {}
W20250815 02:08:53.167981 11063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:53.168164 11063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:53.168198 11063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:53.169613 11063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:53.169659 11063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:53.169682 11063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:53.169699 11063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:53.171216 11063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:43049
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:43049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.11063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:53.171459 11063 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:53.171687 11063 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:53.174093 11071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:53.174263 11063 server_base.cc:1047] running on GCE node
W20250815 02:08:53.174103 11068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:53.174103 11069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:53.174525 11063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:53.174779 11063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:53.175906 11063 hybrid_clock.cc:648] HybridClock initialized: now 1755223733175886 us; error 37 us; skew 500 ppm
I20250815 02:08:53.177027 11063 webserver.cc:480] Webserver started at http://127.6.59.190:46491/ using document root <none> and password file <none>
I20250815 02:08:53.177217 11063 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:53.177256 11063 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:53.177373 11063 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:53.178301 11063 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "b12dbbcdd12843b6b6c35a52c69f978e"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.178612 11063 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "b12dbbcdd12843b6b6c35a52c69f978e"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.179755 11063 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:53.180425 11077 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.180584 11063 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:53.180660 11063 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
uuid: "b12dbbcdd12843b6b6c35a52c69f978e"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.180720 11063 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:53.194535 11063 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:53.194795 11063 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:53.194911 11063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:53.198549 11063 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:43049
I20250815 02:08:53.198612 11129 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:43049 every 8 connection(s)
I20250815 02:08:53.198902 11063 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250815 02:08:53.199481 11130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:53.201721 11130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e: Bootstrap starting.
I20250815 02:08:53.202345 11130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:53.202603 11130 log.cc:826] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:53.203186 11130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e: No bootstrap required, opened a new log
I20250815 02:08:53.204282 11130 raft_consensus.cc:357] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b12dbbcdd12843b6b6c35a52c69f978e" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43049 } }
I20250815 02:08:53.204398 11130 raft_consensus.cc:383] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:53.204432 11130 raft_consensus.cc:738] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b12dbbcdd12843b6b6c35a52c69f978e, State: Initialized, Role: FOLLOWER
I20250815 02:08:53.204576 11130 consensus_queue.cc:260] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b12dbbcdd12843b6b6c35a52c69f978e" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43049 } }
I20250815 02:08:53.204633 11130 raft_consensus.cc:397] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:53.204656 11130 raft_consensus.cc:491] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:53.204704 11130 raft_consensus.cc:3058] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:53.205207 11130 raft_consensus.cc:513] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b12dbbcdd12843b6b6c35a52c69f978e" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43049 } }
I20250815 02:08:53.205294 11130 leader_election.cc:304] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b12dbbcdd12843b6b6c35a52c69f978e; no voters:
I20250815 02:08:53.205428 11130 leader_election.cc:290] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:53.205493 11133 raft_consensus.cc:2802] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:53.205643 11130 sys_catalog.cc:564] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:53.205654 11133 raft_consensus.cc:695] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [term 1 LEADER]: Becoming Leader. State: Replica: b12dbbcdd12843b6b6c35a52c69f978e, State: Running, Role: LEADER
I20250815 02:08:53.205763 11133 consensus_queue.cc:237] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b12dbbcdd12843b6b6c35a52c69f978e" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43049 } }
I20250815 02:08:53.206182 11135 sys_catalog.cc:455] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b12dbbcdd12843b6b6c35a52c69f978e. Latest consensus state: current_term: 1 leader_uuid: "b12dbbcdd12843b6b6c35a52c69f978e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b12dbbcdd12843b6b6c35a52c69f978e" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43049 } } }
I20250815 02:08:53.206166 11134 sys_catalog.cc:455] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b12dbbcdd12843b6b6c35a52c69f978e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b12dbbcdd12843b6b6c35a52c69f978e" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 43049 } } }
I20250815 02:08:53.206364 11134 sys_catalog.cc:458] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:53.206583 11135 sys_catalog.cc:458] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:53.206581 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 11063
I20250815 02:08:53.206670 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250815 02:08:53.206763 11142 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:53.207221 11142 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:53.208648 11142 catalog_manager.cc:1349] Generated new cluster ID: eb3e39ee9e734000a0f85f8eafb44453
I20250815 02:08:53.208709 11142 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:53.216439 11142 catalog_manager.cc:1372] Generated new certificate authority record
I20250815 02:08:53.216934 11142 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:53.221503 11142 catalog_manager.cc:5955] T 00000000000000000000000000000000 P b12dbbcdd12843b6b6c35a52c69f978e: Generated new TSK 0
I20250815 02:08:53.221663 11142 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250815 02:08:53.225715 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.129:0
--local_ip_for_outbound_sockets=127.6.59.129
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43049
--builtin_ntp_servers=127.6.59.148:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250815 02:08:53.303261 11154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:53.303427 11154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:53.303458 11154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:53.304859 11154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:53.304915 11154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.129
I20250815 02:08:53.306496 11154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:43049
--never_fsync=true
--heap_profile_path=/tmp/kudu.11154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.129
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:53.306718 11154 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:53.306973 11154 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:53.309463 11159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:53.309458 11162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:53.309458 11160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:53.310137 11154 server_base.cc:1047] running on GCE node
I20250815 02:08:53.310304 11154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:53.310498 11154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:53.311631 11154 hybrid_clock.cc:648] HybridClock initialized: now 1755223733311609 us; error 40 us; skew 500 ppm
I20250815 02:08:53.312747 11154 webserver.cc:480] Webserver started at http://127.6.59.129:44517/ using document root <none> and password file <none>
I20250815 02:08:53.312937 11154 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:53.312980 11154 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:53.313102 11154 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:53.313928 11154 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "b560c4dbecd14c5e87b3f4e3965dc0f7"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.314261 11154 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "b560c4dbecd14c5e87b3f4e3965dc0f7"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.315373 11154 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:53.316073 11168 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.316252 11154 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:53.316345 11154 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "b560c4dbecd14c5e87b3f4e3965dc0f7"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.316411 11154 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:53.327210 11154 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:53.327482 11154 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:53.327602 11154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:53.327854 11154 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:53.328225 11154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:53.328276 11154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.328315 11154 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:53.328338 11154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.335656 11154 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.129:38449
I20250815 02:08:53.335727 11281 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.129:38449 every 8 connection(s)
I20250815 02:08:53.336083 11154 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250815 02:08:53.340299 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 11154
I20250815 02:08:53.340415 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1755223715463109-6382-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250815 02:08:53.340688 11282 heartbeater.cc:344] Connected to a master server at 127.6.59.190:43049
I20250815 02:08:53.340790 11282 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:53.341003 11282 heartbeater.cc:507] Master 127.6.59.190:43049 requested a full tablet report, sending...
I20250815 02:08:53.341488 11094 ts_manager.cc:194] Registered new tserver with Master: b560c4dbecd14c5e87b3f4e3965dc0f7 (127.6.59.129:38449)
I20250815 02:08:53.342191 11094 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.129:40417
I20250815 02:08:53.352207 6382 external_mini_cluster.cc:949] 1 TS(s) registered with all masters
I20250815 02:08:53.358299 11094 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52020:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20250815 02:08:53.365084 11216 tablet_service.cc:1468] Processing CreateTablet for tablet 0a699ee6797b4065ad2065e01f48930f (DEFAULT_TABLE table=TestTable [id=25eca11a431842d989cac6858414d43f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:53.365376 11216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a699ee6797b4065ad2065e01f48930f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:53.367568 11292 tablet_bootstrap.cc:492] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7: Bootstrap starting.
I20250815 02:08:53.368141 11292 tablet_bootstrap.cc:654] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:53.368356 11292 log.cc:826] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:53.368964 11292 tablet_bootstrap.cc:492] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7: No bootstrap required, opened a new log
I20250815 02:08:53.369025 11292 ts_tablet_manager.cc:1397] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:53.370244 11292 raft_consensus.cc:357] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560c4dbecd14c5e87b3f4e3965dc0f7" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38449 } }
I20250815 02:08:53.370357 11292 raft_consensus.cc:383] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:53.370375 11292 raft_consensus.cc:738] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b560c4dbecd14c5e87b3f4e3965dc0f7, State: Initialized, Role: FOLLOWER
I20250815 02:08:53.370438 11292 consensus_queue.cc:260] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560c4dbecd14c5e87b3f4e3965dc0f7" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38449 } }
I20250815 02:08:53.370482 11292 raft_consensus.cc:397] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:53.370501 11292 raft_consensus.cc:491] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:53.370532 11292 raft_consensus.cc:3058] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:53.370998 11292 raft_consensus.cc:513] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560c4dbecd14c5e87b3f4e3965dc0f7" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38449 } }
I20250815 02:08:53.371086 11292 leader_election.cc:304] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b560c4dbecd14c5e87b3f4e3965dc0f7; no voters:
I20250815 02:08:53.371206 11292 leader_election.cc:290] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:53.371310 11294 raft_consensus.cc:2802] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:53.371423 11282 heartbeater.cc:499] Master 127.6.59.190:43049 was elected leader, sending a full tablet report...
I20250815 02:08:53.371382 11292 ts_tablet_manager.cc:1428] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250815 02:08:53.371553 11294 raft_consensus.cc:695] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [term 1 LEADER]: Becoming Leader. State: Replica: b560c4dbecd14c5e87b3f4e3965dc0f7, State: Running, Role: LEADER
I20250815 02:08:53.371656 11294 consensus_queue.cc:237] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560c4dbecd14c5e87b3f4e3965dc0f7" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38449 } }
I20250815 02:08:53.372665 11094 catalog_manager.cc:5582] T 0a699ee6797b4065ad2065e01f48930f P b560c4dbecd14c5e87b3f4e3965dc0f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b560c4dbecd14c5e87b3f4e3965dc0f7 (127.6.59.129). New cstate: current_term: 1 leader_uuid: "b560c4dbecd14c5e87b3f4e3965dc0f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560c4dbecd14c5e87b3f4e3965dc0f7" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38449 } health_report { overall_health: HEALTHY } } }
I20250815 02:08:53.378795 6382 external_mini_cluster.cc:949] 1 TS(s) registered with all masters
I20250815 02:08:53.379184 6382 ts_itest-base.cc:246] Waiting for 1 tablets on tserver b560c4dbecd14c5e87b3f4e3965dc0f7 to finish bootstrapping
I20250815 02:08:53.527933 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 11154
I20250815 02:08:53.532471 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 11063
2025-08-15T02:08:53Z chronyd exiting
[ OK ] ListTableCliSimpleParamTest.TestListTables/2 (466 ms)
[----------] 1 test from ListTableCliSimpleParamTest (467 ms total)
[----------] 1 test from ListTableCliParamTest
[ RUN ] ListTableCliParamTest.ListTabletWithPartitionInfo/4
I20250815 02:08:53.550817 6382 test_util.cc:276] Using random seed: -551305508
[ OK ] ListTableCliParamTest.ListTabletWithPartitionInfo/4 (1 ms)
[----------] 1 test from ListTableCliParamTest (1 ms total)
[----------] 1 test from IsSecure/SecureClusterAdminCliParamTest
[ RUN ] IsSecure/SecureClusterAdminCliParamTest.TestRebuildMaster/0
2025-08-15T02:08:53Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-15T02:08:53Z Disabled control of system clock
I20250815 02:08:53.559152 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:42237
--webserver_interface=127.6.59.190
--webserver_port=0
--builtin_ntp_servers=127.6.59.148:43943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:42237 with env {}
W20250815 02:08:53.636500 11316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:53.636688 11316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:53.636708 11316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:53.638137 11316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:53.638177 11316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:53.638190 11316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:53.638202 11316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:53.639731 11316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:43943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:42237
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:42237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.11316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:53.639902 11316 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:53.640089 11316 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:53.642381 11322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:53.642370 11324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:53.642374 11321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:53.642596 11316 server_base.cc:1047] running on GCE node
I20250815 02:08:53.642860 11316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:53.643083 11316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:53.644219 11316 hybrid_clock.cc:648] HybridClock initialized: now 1755223733644200 us; error 32 us; skew 500 ppm
I20250815 02:08:53.645352 11316 webserver.cc:480] Webserver started at http://127.6.59.190:44571/ using document root <none> and password file <none>
I20250815 02:08:53.645540 11316 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:53.645594 11316 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:53.645713 11316 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:53.646612 11316 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/instance:
uuid: "5cdf5185a61a44eba6bf235716619c3d"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.646906 11316 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal/instance:
uuid: "5cdf5185a61a44eba6bf235716619c3d"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.648038 11316 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:53.648698 11330 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.648867 11316 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:53.648939 11316 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
uuid: "5cdf5185a61a44eba6bf235716619c3d"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.648999 11316 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:53.668821 11316 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:53.669112 11316 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:53.669229 11316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:53.673159 11316 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:42237
I20250815 02:08:53.673242 11382 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:42237 every 8 connection(s)
I20250815 02:08:53.673558 11316 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/info.pb
I20250815 02:08:53.674171 11383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:53.676401 11383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d: Bootstrap starting.
I20250815 02:08:53.676990 11383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:53.677242 11383 log.cc:826] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:53.677851 11383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d: No bootstrap required, opened a new log
I20250815 02:08:53.678987 11383 raft_consensus.cc:357] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5185a61a44eba6bf235716619c3d" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } }
I20250815 02:08:53.679109 11383 raft_consensus.cc:383] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:53.679131 11383 raft_consensus.cc:738] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdf5185a61a44eba6bf235716619c3d, State: Initialized, Role: FOLLOWER
I20250815 02:08:53.679256 11383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5185a61a44eba6bf235716619c3d" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } }
I20250815 02:08:53.679312 11383 raft_consensus.cc:397] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:53.679348 11383 raft_consensus.cc:491] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:53.679392 11383 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:53.679915 11383 raft_consensus.cc:513] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5185a61a44eba6bf235716619c3d" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } }
I20250815 02:08:53.680004 11383 leader_election.cc:304] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cdf5185a61a44eba6bf235716619c3d; no voters:
I20250815 02:08:53.680156 11383 leader_election.cc:290] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:53.680226 11386 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:53.680361 11386 raft_consensus.cc:695] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [term 1 LEADER]: Becoming Leader. State: Replica: 5cdf5185a61a44eba6bf235716619c3d, State: Running, Role: LEADER
I20250815 02:08:53.680361 11383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:53.680464 11386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5185a61a44eba6bf235716619c3d" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } }
I20250815 02:08:53.680791 11387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cdf5185a61a44eba6bf235716619c3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5185a61a44eba6bf235716619c3d" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } } }
I20250815 02:08:53.680827 11388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cdf5185a61a44eba6bf235716619c3d. Latest consensus state: current_term: 1 leader_uuid: "5cdf5185a61a44eba6bf235716619c3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5185a61a44eba6bf235716619c3d" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } } }
I20250815 02:08:53.681053 11387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:53.681061 11388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:53.681344 11396 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:53.681820 11396 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:53.683029 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 11316
I20250815 02:08:53.683142 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal/instance
I20250815 02:08:53.683487 11396 catalog_manager.cc:1349] Generated new cluster ID: 05ec429a42234624a8173e21e340dc2e
I20250815 02:08:53.683568 11396 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:53.694365 11396 catalog_manager.cc:1372] Generated new certificate authority record
I20250815 02:08:53.694854 11396 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:53.701539 11396 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 5cdf5185a61a44eba6bf235716619c3d: Generated new TSK 0
I20250815 02:08:53.701736 11396 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250815 02:08:53.711088 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.129:0
--local_ip_for_outbound_sockets=127.6.59.129
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42237
--builtin_ntp_servers=127.6.59.148:43943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20250815 02:08:53.789197 11407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:53.789369 11407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:53.789388 11407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:53.790843 11407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:53.790894 11407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.129
I20250815 02:08:53.792315 11407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:43943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.59.129
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42237
--never_fsync=true
--heap_profile_path=/tmp/kudu.11407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.129
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:53.792512 11407 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:53.792745 11407 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:53.795123 11412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:53.795135 11413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:53.795244 11407 server_base.cc:1047] running on GCE node
W20250815 02:08:53.795135 11415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:53.795471 11407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:53.795670 11407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:53.796806 11407 hybrid_clock.cc:648] HybridClock initialized: now 1755223733796792 us; error 26 us; skew 500 ppm
I20250815 02:08:53.797953 11407 webserver.cc:480] Webserver started at http://127.6.59.129:44017/ using document root <none> and password file <none>
I20250815 02:08:53.798197 11407 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:53.798242 11407 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:53.798360 11407 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:53.799201 11407 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/instance:
uuid: "f4354725d6514d32b8f4a8bebd6a9d94"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.799496 11407 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal/instance:
uuid: "f4354725d6514d32b8f4a8bebd6a9d94"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.800652 11407 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20250815 02:08:53.801337 11421 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.801517 11407 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:53.801611 11407 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal
uuid: "f4354725d6514d32b8f4a8bebd6a9d94"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.801676 11407 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:53.824429 11407 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:53.824735 11407 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:53.824860 11407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:53.825109 11407 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:53.825485 11407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:53.825536 11407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.825585 11407 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:53.825609 11407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.832212 11407 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.129:38827
I20250815 02:08:53.832288 11534 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.129:38827 every 8 connection(s)
I20250815 02:08:53.832589 11407 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/info.pb
I20250815 02:08:53.835670 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 11407
I20250815 02:08:53.835768 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal/instance
I20250815 02:08:53.836984 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.130:0
--local_ip_for_outbound_sockets=127.6.59.130
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42237
--builtin_ntp_servers=127.6.59.148:43943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250815 02:08:53.837406 11535 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42237
I20250815 02:08:53.837503 11535 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:53.837702 11535 heartbeater.cc:507] Master 127.6.59.190:42237 requested a full tablet report, sending...
I20250815 02:08:53.838186 11347 ts_manager.cc:194] Registered new tserver with Master: f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129:38827)
I20250815 02:08:53.838887 11347 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.129:51777
W20250815 02:08:53.915237 11538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:53.915410 11538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:53.915441 11538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:53.916754 11538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:53.916810 11538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.130
I20250815 02:08:53.918344 11538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:43943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.59.130
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42237
--never_fsync=true
--heap_profile_path=/tmp/kudu.11538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.130
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:53.918574 11538 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:53.918790 11538 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:53.921201 11543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:53.921196 11544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:53.921345 11538 server_base.cc:1047] running on GCE node
W20250815 02:08:53.921196 11546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:53.921532 11538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:53.921752 11538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:53.922951 11538 hybrid_clock.cc:648] HybridClock initialized: now 1755223733922933 us; error 28 us; skew 500 ppm
I20250815 02:08:53.924098 11538 webserver.cc:480] Webserver started at http://127.6.59.130:38625/ using document root <none> and password file <none>
I20250815 02:08:53.924290 11538 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:53.924336 11538 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:53.924461 11538 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:53.925318 11538 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/instance:
uuid: "93ce45ca80c8401e9cbb10f379397239"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.925594 11538 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal/instance:
uuid: "93ce45ca80c8401e9cbb10f379397239"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.926796 11538 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250815 02:08:53.927479 11552 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.927642 11538 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:53.927711 11538 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal
uuid: "93ce45ca80c8401e9cbb10f379397239"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:53.927765 11538 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:53.935987 11538 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:53.936193 11538 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:53.936297 11538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:53.936479 11538 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:53.936745 11538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:53.936776 11538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.936810 11538 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:53.936844 11538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:53.942262 11538 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.130:45727
I20250815 02:08:53.942341 11665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.130:45727 every 8 connection(s)
I20250815 02:08:53.942598 11538 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/info.pb
I20250815 02:08:53.947016 11666 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42237
I20250815 02:08:53.947100 11666 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:53.947285 11666 heartbeater.cc:507] Master 127.6.59.190:42237 requested a full tablet report, sending...
I20250815 02:08:53.947618 11347 ts_manager.cc:194] Registered new tserver with Master: 93ce45ca80c8401e9cbb10f379397239 (127.6.59.130:45727)
I20250815 02:08:53.947960 11347 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.130:55757
I20250815 02:08:53.951692 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 11538
I20250815 02:08:53.951768 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal/instance
I20250815 02:08:53.952739 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.131:0
--local_ip_for_outbound_sockets=127.6.59.131
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42237
--builtin_ntp_servers=127.6.59.148:43943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20250815 02:08:54.029542 11669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:54.029708 11669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:54.029727 11669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:54.031100 11669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:54.031147 11669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.131
I20250815 02:08:54.032575 11669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:43943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.59.131
--webserver_port=0
--tserver_master_addrs=127.6.59.190:42237
--never_fsync=true
--heap_profile_path=/tmp/kudu.11669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.131
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:54.032742 11669 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:54.032930 11669 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:54.035434 11677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:54.035454 11669 server_base.cc:1047] running on GCE node
W20250815 02:08:54.035423 11674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:54.035419 11675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:54.035799 11669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:54.035971 11669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:54.037092 11669 hybrid_clock.cc:648] HybridClock initialized: now 1755223734037079 us; error 30 us; skew 500 ppm
I20250815 02:08:54.038151 11669 webserver.cc:480] Webserver started at http://127.6.59.131:46337/ using document root <none> and password file <none>
I20250815 02:08:54.038345 11669 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:54.038383 11669 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:54.038496 11669 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:54.039314 11669 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/instance:
uuid: "2a31cfc88db341028d33e8e4cd42ae0c"
format_stamp: "Formatted at 2025-08-15 02:08:54 on dist-test-slave-wq2q"
I20250815 02:08:54.039623 11669 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal/instance:
uuid: "2a31cfc88db341028d33e8e4cd42ae0c"
format_stamp: "Formatted at 2025-08-15 02:08:54 on dist-test-slave-wq2q"
I20250815 02:08:54.040726 11669 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20250815 02:08:54.041366 11683 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:54.041513 11669 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:54.041610 11669 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal
uuid: "2a31cfc88db341028d33e8e4cd42ae0c"
format_stamp: "Formatted at 2025-08-15 02:08:54 on dist-test-slave-wq2q"
I20250815 02:08:54.041656 11669 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:54.058027 11669 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:54.058256 11669 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:54.058343 11669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:54.058518 11669 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:54.058784 11669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250815 02:08:54.058812 11669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:54.058838 11669 ts_tablet_manager.cc:610] Registered 0 tablets
I20250815 02:08:54.058856 11669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:54.064316 11669 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.131:33489
I20250815 02:08:54.064366 11796 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.131:33489 every 8 connection(s)
I20250815 02:08:54.064668 11669 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/info.pb
I20250815 02:08:54.067164 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 11669
I20250815 02:08:54.067248 6382 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal/instance
I20250815 02:08:54.069012 11797 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42237
I20250815 02:08:54.069092 11797 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:54.069288 11797 heartbeater.cc:507] Master 127.6.59.190:42237 requested a full tablet report, sending...
I20250815 02:08:54.069600 11347 ts_manager.cc:194] Registered new tserver with Master: 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131:33489)
I20250815 02:08:54.069947 11347 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.131:55841
I20250815 02:08:54.079042 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:54.083716 6382 test_util.cc:276] Using random seed: -550772606
I20250815 02:08:54.089073 11347 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41894:
name: "pre_rebuild"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250815 02:08:54.089321 11347 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table pre_rebuild in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250815 02:08:54.095227 11731 tablet_service.cc:1468] Processing CreateTablet for tablet e48639cfbf5f418e8de934daff9e77da (DEFAULT_TABLE table=pre_rebuild [id=0e233d6421f6476f92d4297e8f6d5726]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:54.095551 11731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e48639cfbf5f418e8de934daff9e77da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:54.095502 11600 tablet_service.cc:1468] Processing CreateTablet for tablet e48639cfbf5f418e8de934daff9e77da (DEFAULT_TABLE table=pre_rebuild [id=0e233d6421f6476f92d4297e8f6d5726]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:54.095757 11600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e48639cfbf5f418e8de934daff9e77da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:54.095762 11469 tablet_service.cc:1468] Processing CreateTablet for tablet e48639cfbf5f418e8de934daff9e77da (DEFAULT_TABLE table=pre_rebuild [id=0e233d6421f6476f92d4297e8f6d5726]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:54.096014 11469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e48639cfbf5f418e8de934daff9e77da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:54.097687 11816 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Bootstrap starting.
I20250815 02:08:54.098343 11817 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Bootstrap starting.
I20250815 02:08:54.098485 11818 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Bootstrap starting.
I20250815 02:08:54.098569 11816 tablet_bootstrap.cc:654] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:54.098920 11816 log.cc:826] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:54.099052 11817 tablet_bootstrap.cc:654] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:54.099215 11818 tablet_bootstrap.cc:654] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:54.099298 11817 log.cc:826] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:54.099491 11818 log.cc:826] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:54.099550 11816 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: No bootstrap required, opened a new log
I20250815 02:08:54.099597 11816 ts_tablet_manager.cc:1397] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:54.099920 11817 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: No bootstrap required, opened a new log
I20250815 02:08:54.099972 11817 ts_tablet_manager.cc:1397] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:54.100101 11818 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: No bootstrap required, opened a new log
I20250815 02:08:54.100165 11818 ts_tablet_manager.cc:1397] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:54.101003 11816 raft_consensus.cc:357] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:54.101135 11816 raft_consensus.cc:383] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:54.101161 11816 raft_consensus.cc:738] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ce45ca80c8401e9cbb10f379397239, State: Initialized, Role: FOLLOWER
I20250815 02:08:54.101171 11817 raft_consensus.cc:357] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:54.101240 11816 consensus_queue.cc:260] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:54.101279 11817 raft_consensus.cc:383] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:54.101303 11817 raft_consensus.cc:738] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a31cfc88db341028d33e8e4cd42ae0c, State: Initialized, Role: FOLLOWER
I20250815 02:08:54.101366 11817 consensus_queue.cc:260] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:54.101436 11816 ts_tablet_manager.cc:1428] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250815 02:08:54.101552 11666 heartbeater.cc:499] Master 127.6.59.190:42237 was elected leader, sending a full tablet report...
I20250815 02:08:54.101557 11817 ts_tablet_manager.cc:1428] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250815 02:08:54.101581 11797 heartbeater.cc:499] Master 127.6.59.190:42237 was elected leader, sending a full tablet report...
I20250815 02:08:54.101691 11818 raft_consensus.cc:357] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:54.101816 11818 raft_consensus.cc:383] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:54.101855 11818 raft_consensus.cc:738] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4354725d6514d32b8f4a8bebd6a9d94, State: Initialized, Role: FOLLOWER
I20250815 02:08:54.101951 11818 consensus_queue.cc:260] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:54.102185 11818 ts_tablet_manager.cc:1428] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250815 02:08:54.102411 11535 heartbeater.cc:499] Master 127.6.59.190:42237 was elected leader, sending a full tablet report...
W20250815 02:08:54.193099 11667 tablet.cc:2378] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250815 02:08:54.315163 11798 tablet.cc:2378] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250815 02:08:54.333290 11536 tablet.cc:2378] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250815 02:08:54.468705 11824 raft_consensus.cc:491] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250815 02:08:54.468843 11824 raft_consensus.cc:513] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:54.469144 11824 leader_election.cc:290] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131:33489), 93ce45ca80c8401e9cbb10f379397239 (127.6.59.130:45727)
I20250815 02:08:54.472802 11620 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "e48639cfbf5f418e8de934daff9e77da" candidate_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ce45ca80c8401e9cbb10f379397239" is_pre_election: true
I20250815 02:08:54.472811 11751 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "e48639cfbf5f418e8de934daff9e77da" candidate_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" is_pre_election: true
I20250815 02:08:54.472976 11620 raft_consensus.cc:2466] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4354725d6514d32b8f4a8bebd6a9d94 in term 0.
I20250815 02:08:54.472976 11751 raft_consensus.cc:2466] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4354725d6514d32b8f4a8bebd6a9d94 in term 0.
I20250815 02:08:54.473198 11422 leader_election.cc:304] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a31cfc88db341028d33e8e4cd42ae0c, f4354725d6514d32b8f4a8bebd6a9d94; no voters:
I20250815 02:08:54.473366 11824 raft_consensus.cc:2802] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250815 02:08:54.473428 11824 raft_consensus.cc:491] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250815 02:08:54.473457 11824 raft_consensus.cc:3058] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:54.474159 11824 raft_consensus.cc:513] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:54.474289 11824 leader_election.cc:290] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [CANDIDATE]: Term 1 election: Requested vote from peers 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131:33489), 93ce45ca80c8401e9cbb10f379397239 (127.6.59.130:45727)
I20250815 02:08:54.474467 11751 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "e48639cfbf5f418e8de934daff9e77da" candidate_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a31cfc88db341028d33e8e4cd42ae0c"
I20250815 02:08:54.474467 11620 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "e48639cfbf5f418e8de934daff9e77da" candidate_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ce45ca80c8401e9cbb10f379397239"
I20250815 02:08:54.474547 11751 raft_consensus.cc:3058] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:54.474547 11620 raft_consensus.cc:3058] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:54.475335 11620 raft_consensus.cc:2466] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4354725d6514d32b8f4a8bebd6a9d94 in term 1.
I20250815 02:08:54.475335 11751 raft_consensus.cc:2466] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4354725d6514d32b8f4a8bebd6a9d94 in term 1.
I20250815 02:08:54.475533 11425 leader_election.cc:304] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93ce45ca80c8401e9cbb10f379397239, f4354725d6514d32b8f4a8bebd6a9d94; no voters:
I20250815 02:08:54.475633 11824 raft_consensus.cc:2802] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:54.475806 11824 raft_consensus.cc:695] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 1 LEADER]: Becoming Leader. State: Replica: f4354725d6514d32b8f4a8bebd6a9d94, State: Running, Role: LEADER
I20250815 02:08:54.475924 11824 consensus_queue.cc:237] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:54.476725 11346 catalog_manager.cc:5582] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129). New cstate: current_term: 1 leader_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } health_report { overall_health: HEALTHY } } }
I20250815 02:08:54.540606 11751 raft_consensus.cc:1273] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 1 FOLLOWER]: Refusing update from remote peer f4354725d6514d32b8f4a8bebd6a9d94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250815 02:08:54.540606 11620 raft_consensus.cc:1273] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Refusing update from remote peer f4354725d6514d32b8f4a8bebd6a9d94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250815 02:08:54.540946 11827 consensus_queue.cc:1035] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:54.541095 11824 consensus_queue.cc:1035] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:54.543902 11834 mvcc.cc:204] Tried to move back new op lower bound from 7189396416677851136 to 7189396416414085120. Current Snapshot: MvccSnapshot[applied={T|T < 7189396416675807232}]
I20250815 02:08:54.546568 11836 mvcc.cc:204] Tried to move back new op lower bound from 7189396416678834176 to 7189396416414085120. Current Snapshot: MvccSnapshot[applied={T|T < 7189396416675807232 or (T in {7189396416677851136})}]
I20250815 02:08:54.699105 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 11316
W20250815 02:08:54.785640 11858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:54.785828 11858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:54.787228 11858 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20250815 02:08:54.805577 11858 minidump.cc:252] Setting minidump size limit to 20M
I20250815 02:08:54.805876 11858 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:54.806085 11858 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:54.808377 11880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:54.808382 11879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:54.808503 11858 server_base.cc:1047] running on GCE node
W20250815 02:08:54.808637 11882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:54.808848 11858 hybrid_clock.cc:584] initializing the hybrid clock with 'system' time source
I20250815 02:08:54.808917 11858 hybrid_clock.cc:648] HybridClock initialized: now 1755223734808916 us; error 98073 us; skew 500 ppm
I20250815 02:08:54.809747 11858 webserver.cc:480] Webserver started at http://0.0.0.0:43577/ using document root <none> and password file <none>
I20250815 02:08:54.809935 11858 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:54.810025 11858 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:54.810137 11858 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250815 02:08:54.810986 11858 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/instance:
uuid: "5ac491abad3145648f2f2eb3c30f220a"
format_stamp: "Formatted at 2025-08-15 02:08:54 on dist-test-slave-wq2q"
I20250815 02:08:54.811280 11858 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal/instance:
uuid: "5ac491abad3145648f2f2eb3c30f220a"
format_stamp: "Formatted at 2025-08-15 02:08:54 on dist-test-slave-wq2q"
I20250815 02:08:54.812433 11858 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250815 02:08:54.813071 11887 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:54.813231 11858 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:54.813320 11858 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
uuid: "5ac491abad3145648f2f2eb3c30f220a"
format_stamp: "Formatted at 2025-08-15 02:08:54 on dist-test-slave-wq2q"
I20250815 02:08:54.813383 11858 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:54.969419 11858 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:54.969758 11858 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:54.969898 11858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:54.970395 11858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:54.972580 11858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: Bootstrap starting.
I20250815 02:08:54.973165 11858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:54.973419 11858 log.cc:826] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:54.974066 11858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: No bootstrap required, opened a new log
I20250815 02:08:54.975327 11858 raft_consensus.cc:357] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER }
I20250815 02:08:54.975435 11858 raft_consensus.cc:383] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:54.975468 11858 raft_consensus.cc:738] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac491abad3145648f2f2eb3c30f220a, State: Initialized, Role: FOLLOWER
I20250815 02:08:54.975571 11858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER }
I20250815 02:08:54.975643 11858 raft_consensus.cc:397] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:54.975685 11858 raft_consensus.cc:491] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:54.975737 11858 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:54.976312 11858 raft_consensus.cc:513] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER }
I20250815 02:08:54.976418 11858 leader_election.cc:304] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ac491abad3145648f2f2eb3c30f220a; no voters:
I20250815 02:08:54.976585 11858 leader_election.cc:290] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [CANDIDATE]: Term 1 election: Requested vote from peers
I20250815 02:08:54.976675 11894 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:54.976814 11894 raft_consensus.cc:695] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 1 LEADER]: Becoming Leader. State: Replica: 5ac491abad3145648f2f2eb3c30f220a, State: Running, Role: LEADER
I20250815 02:08:54.976900 11894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER }
I20250815 02:08:54.977309 11895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ac491abad3145648f2f2eb3c30f220a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER } }
I20250815 02:08:54.977293 11896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ac491abad3145648f2f2eb3c30f220a. Latest consensus state: current_term: 1 leader_uuid: "5ac491abad3145648f2f2eb3c30f220a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER } }
I20250815 02:08:54.977401 11895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:54.977406 11896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:54.978538 11858 tablet_replica.cc:331] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: stopping tablet replica
I20250815 02:08:54.978611 11858 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 1 LEADER]: Raft consensus shutting down.
I20250815 02:08:54.978647 11858 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250815 02:08:54.978950 11858 master.cc:561] Master@0.0.0.0:7051 shutting down...
W20250815 02:08:54.979027 11858 acceptor_pool.cc:194] Could not shut down acceptor socket on 0.0.0.0:7051: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20250815 02:08:54.989838 11858 master.cc:583] Master@0.0.0.0:7051 shutdown complete.
I20250815 02:08:54.996186 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 11407
I20250815 02:08:55.001995 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 11538
I20250815 02:08:55.006451 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 11669
I20250815 02:08:55.011241 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:42237
--webserver_interface=127.6.59.190
--webserver_port=44571
--builtin_ntp_servers=127.6.59.148:43943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.59.190:42237 with env {}
W20250815 02:08:55.089573 11900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:55.089768 11900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:55.089788 11900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:55.091214 11900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250815 02:08:55.091259 11900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:55.091274 11900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250815 02:08:55.091285 11900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250815 02:08:55.092767 11900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:43943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.59.190:42237
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.59.190:42237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.59.190
--webserver_port=44571
--never_fsync=true
--heap_profile_path=/tmp/kudu.11900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:55.092975 11900 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:55.093173 11900 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:55.095585 11908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:55.095597 11905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:55.095762 11900 server_base.cc:1047] running on GCE node
W20250815 02:08:55.095598 11906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:55.096035 11900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:55.096264 11900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:55.097391 11900 hybrid_clock.cc:648] HybridClock initialized: now 1755223735097383 us; error 27 us; skew 500 ppm
I20250815 02:08:55.098567 11900 webserver.cc:480] Webserver started at http://127.6.59.190:44571/ using document root <none> and password file <none>
I20250815 02:08:55.098781 11900 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:55.098850 11900 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:55.100114 11900 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20250815 02:08:55.100777 11914 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:55.100924 11900 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250815 02:08:55.100992 11900 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
uuid: "5ac491abad3145648f2f2eb3c30f220a"
format_stamp: "Formatted at 2025-08-15 02:08:54 on dist-test-slave-wq2q"
I20250815 02:08:55.101291 11900 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:55.111011 11900 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:55.111289 11900 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:55.111402 11900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:55.115365 11900 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.190:42237
I20250815 02:08:55.115430 11966 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.190:42237 every 8 connection(s)
I20250815 02:08:55.115751 11900 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/info.pb
I20250815 02:08:55.116832 11967 sys_catalog.cc:263] Verifying existing consensus state
I20250815 02:08:55.118425 11967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: Bootstrap starting.
I20250815 02:08:55.119763 11967 log.cc:826] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:55.121061 11967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=2 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:55.121294 11967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: Bootstrap complete.
I20250815 02:08:55.122499 11967 raft_consensus.cc:357] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } }
I20250815 02:08:55.122617 11967 raft_consensus.cc:738] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac491abad3145648f2f2eb3c30f220a, State: Initialized, Role: FOLLOWER
I20250815 02:08:55.122727 11967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } }
I20250815 02:08:55.122790 11967 raft_consensus.cc:397] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250815 02:08:55.122818 11967 raft_consensus.cc:491] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250815 02:08:55.122870 11967 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:55.123376 11967 raft_consensus.cc:513] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } }
I20250815 02:08:55.123462 11967 leader_election.cc:304] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ac491abad3145648f2f2eb3c30f220a; no voters:
I20250815 02:08:55.123600 11967 leader_election.cc:290] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [CANDIDATE]: Term 2 election: Requested vote from peers
I20250815 02:08:55.123660 11970 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 2 FOLLOWER]: Leader election won for term 2
I20250815 02:08:55.123826 11970 raft_consensus.cc:695] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [term 2 LEADER]: Becoming Leader. State: Replica: 5ac491abad3145648f2f2eb3c30f220a, State: Running, Role: LEADER
I20250815 02:08:55.123811 11967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [sys.catalog]: configured and running, proceeding with master startup.
I20250815 02:08:55.123973 11970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } }
I20250815 02:08:55.124293 11972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ac491abad3145648f2f2eb3c30f220a. Latest consensus state: current_term: 2 leader_uuid: "5ac491abad3145648f2f2eb3c30f220a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } } }
I20250815 02:08:55.124420 11972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:55.124322 11971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5ac491abad3145648f2f2eb3c30f220a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac491abad3145648f2f2eb3c30f220a" member_type: VOTER last_known_addr { host: "127.6.59.190" port: 42237 } } }
I20250815 02:08:55.124624 11971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a [sys.catalog]: This master's current role is: LEADER
I20250815 02:08:55.124676 11977 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250815 02:08:55.125241 11977 catalog_manager.cc:671] Loaded metadata for table pre_rebuild [id=6cc9238e07034c9f994018ff481ef09a]
I20250815 02:08:55.125430 11977 tablet_loader.cc:96] loaded metadata for tablet e48639cfbf5f418e8de934daff9e77da (table pre_rebuild [id=6cc9238e07034c9f994018ff481ef09a])
I20250815 02:08:55.125483 11977 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250815 02:08:55.125594 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 11900
I20250815 02:08:55.125913 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.129:38827
--local_ip_for_outbound_sockets=127.6.59.129
--tserver_master_addrs=127.6.59.190:42237
--webserver_port=44017
--webserver_interface=127.6.59.129
--builtin_ntp_servers=127.6.59.148:43943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250815 02:08:55.126766 11977 catalog_manager.cc:1349] Generated new cluster ID: 8a4d9ee3b16f4c0bac5c41627e7298fa
I20250815 02:08:55.126816 11977 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250815 02:08:55.132586 11977 catalog_manager.cc:1372] Generated new certificate authority record
I20250815 02:08:55.133325 11977 catalog_manager.cc:1506] Loading token signing keys...
I20250815 02:08:55.141206 11977 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: Generated new TSK 0
I20250815 02:08:55.141413 11977 catalog_manager.cc:1516] Initializing in-progress tserver states...
W20250815 02:08:55.203723 11989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:55.203913 11989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:55.203933 11989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:55.205374 11989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:55.205426 11989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.129
I20250815 02:08:55.206945 11989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:43943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.129:38827
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.59.129
--webserver_port=44017
--tserver_master_addrs=127.6.59.190:42237
--never_fsync=true
--heap_profile_path=/tmp/kudu.11989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.129
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:55.207130 11989 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:55.207345 11989 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:55.209756 11994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:55.209897 11989 server_base.cc:1047] running on GCE node
W20250815 02:08:55.209731 11997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:55.209744 11995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:55.210191 11989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:55.210386 11989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:55.211517 11989 hybrid_clock.cc:648] HybridClock initialized: now 1755223735211495 us; error 40 us; skew 500 ppm
I20250815 02:08:55.212595 11989 webserver.cc:480] Webserver started at http://127.6.59.129:44017/ using document root <none> and password file <none>
I20250815 02:08:55.212774 11989 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:55.212833 11989 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:55.214063 11989 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:55.214741 12003 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:55.214913 11989 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:55.215003 11989 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal
uuid: "f4354725d6514d32b8f4a8bebd6a9d94"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:55.215344 11989 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:55.250077 11989 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:55.250373 11989 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:55.250490 11989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:55.250741 11989 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:55.251245 12010 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250815 02:08:55.252115 11989 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250815 02:08:55.252177 11989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:55.252213 11989 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250815 02:08:55.252965 11989 ts_tablet_manager.cc:610] Registered 1 tablets
I20250815 02:08:55.253011 11989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:55.253062 12010 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Bootstrap starting.
I20250815 02:08:55.260422 11989 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.129:38827
I20250815 02:08:55.260470 12117 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.129:38827 every 8 connection(s)
I20250815 02:08:55.260903 11989 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/info.pb
I20250815 02:08:55.268455 12010 log.cc:826] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:55.268579 12118 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42237
I20250815 02:08:55.268679 12118 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:55.268891 12118 heartbeater.cc:507] Master 127.6.59.190:42237 requested a full tablet report, sending...
I20250815 02:08:55.269660 11931 ts_manager.cc:194] Registered new tserver with Master: f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129:38827)
I20250815 02:08:55.270715 11931 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.129:59991
I20250815 02:08:55.271085 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 11989
I20250815 02:08:55.271338 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.130:45727
--local_ip_for_outbound_sockets=127.6.59.130
--tserver_master_addrs=127.6.59.190:42237
--webserver_port=38625
--webserver_interface=127.6.59.130
--builtin_ntp_servers=127.6.59.148:43943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250815 02:08:55.299300 12010 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Bootstrap replayed 1/1 log segments. Stats: ops{read=206 overwritten=0 applied=206 ignored=0} inserts{seen=10250 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:55.299645 12010 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Bootstrap complete.
I20250815 02:08:55.300580 12010 ts_tablet_manager.cc:1397] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Time spent bootstrapping tablet: real 0.048s user 0.030s sys 0.015s
I20250815 02:08:55.301434 12010 raft_consensus.cc:357] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:55.301630 12010 raft_consensus.cc:738] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4354725d6514d32b8f4a8bebd6a9d94, State: Initialized, Role: FOLLOWER
I20250815 02:08:55.301759 12010 consensus_queue.cc:260] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 206, Last appended: 1.206, Last appended by leader: 206, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:55.302089 12010 ts_tablet_manager.cc:1428] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Time spent starting tablet: real 0.001s user 0.003s sys 0.001s
I20250815 02:08:55.302136 12118 heartbeater.cc:499] Master 127.6.59.190:42237 was elected leader, sending a full tablet report...
W20250815 02:08:55.362356 12123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:55.362534 12123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:55.362555 12123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:55.363986 12123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:55.364034 12123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.130
I20250815 02:08:55.365479 12123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:43943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.130:45727
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.59.130
--webserver_port=38625
--tserver_master_addrs=127.6.59.190:42237
--never_fsync=true
--heap_profile_path=/tmp/kudu.12123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.130
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:55.365664 12123 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:55.365864 12123 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:55.368405 12129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:55.368420 12132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:55.368423 12130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:55.368556 12123 server_base.cc:1047] running on GCE node
I20250815 02:08:55.368808 12123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:55.368995 12123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:55.370121 12123 hybrid_clock.cc:648] HybridClock initialized: now 1755223735370113 us; error 27 us; skew 500 ppm
I20250815 02:08:55.371240 12123 webserver.cc:480] Webserver started at http://127.6.59.130:38625/ using document root <none> and password file <none>
I20250815 02:08:55.371431 12123 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:55.371490 12123 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:55.372697 12123 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:55.373346 12138 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:55.373529 12123 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:55.373602 12123 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal
uuid: "93ce45ca80c8401e9cbb10f379397239"
format_stamp: "Formatted at 2025-08-15 02:08:53 on dist-test-slave-wq2q"
I20250815 02:08:55.373844 12123 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:55.401142 12123 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:55.401367 12123 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:55.401453 12123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:55.401657 12123 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:55.402135 12145 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250815 02:08:55.402973 12123 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250815 02:08:55.403018 12123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:55.403052 12123 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250815 02:08:55.403625 12123 ts_tablet_manager.cc:610] Registered 1 tablets
I20250815 02:08:55.403657 12123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:55.403718 12145 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Bootstrap starting.
I20250815 02:08:55.410169 12123 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.130:45727
I20250815 02:08:55.410229 12252 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.130:45727 every 8 connection(s)
I20250815 02:08:55.410530 12123 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/info.pb
I20250815 02:08:55.415169 12253 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42237
I20250815 02:08:55.415256 12253 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:55.415428 12253 heartbeater.cc:507] Master 127.6.59.190:42237 requested a full tablet report, sending...
I20250815 02:08:55.415954 11931 ts_manager.cc:194] Registered new tserver with Master: 93ce45ca80c8401e9cbb10f379397239 (127.6.59.130:45727)
I20250815 02:08:55.416317 11931 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.130:46755
I20250815 02:08:55.417384 12145 log.cc:826] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:55.420684 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 12123
I20250815 02:08:55.420994 6382 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
/tmp/dist-test-taskpPPZ_f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.59.131:33489
--local_ip_for_outbound_sockets=127.6.59.131
--tserver_master_addrs=127.6.59.190:42237
--webserver_port=46337
--webserver_interface=127.6.59.131
--builtin_ntp_servers=127.6.59.148:43943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250815 02:08:55.446759 12145 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Bootstrap replayed 1/1 log segments. Stats: ops{read=206 overwritten=0 applied=206 ignored=0} inserts{seen=10250 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:55.447091 12145 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Bootstrap complete.
I20250815 02:08:55.447996 12145 ts_tablet_manager.cc:1397] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Time spent bootstrapping tablet: real 0.044s user 0.027s sys 0.016s
I20250815 02:08:55.448908 12145 raft_consensus.cc:357] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:55.449092 12145 raft_consensus.cc:738] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ce45ca80c8401e9cbb10f379397239, State: Initialized, Role: FOLLOWER
I20250815 02:08:55.449185 12145 consensus_queue.cc:260] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 206, Last appended: 1.206, Last appended by leader: 206, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:55.449466 12145 ts_tablet_manager.cc:1428] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20250815 02:08:55.449523 12253 heartbeater.cc:499] Master 127.6.59.190:42237 was elected leader, sending a full tablet report...
W20250815 02:08:55.501624 12257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250815 02:08:55.501792 12257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250815 02:08:55.501816 12257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250815 02:08:55.503177 12257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250815 02:08:55.503233 12257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.59.131
I20250815 02:08:55.504726 12257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.59.148:43943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.59.131:33489
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.59.131
--webserver_port=46337
--tserver_master_addrs=127.6.59.190:42237
--never_fsync=true
--heap_profile_path=/tmp/kudu.12257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.59.131
--log_dir=/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 6307ab29b78ebc064971e01ca1da31590298dcda
build type RELEASE
built by None at 15 Aug 2025 01:43:13 UTC on 5fd53c4cbb9d
build id 7571
I20250815 02:08:55.504920 12257 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250815 02:08:55.505106 12257 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250815 02:08:55.507411 12263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250815 02:08:55.507434 12266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:55.507632 12257 server_base.cc:1047] running on GCE node
W20250815 02:08:55.507414 12264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250815 02:08:55.507846 12257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250815 02:08:55.508041 12257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250815 02:08:55.509173 12257 hybrid_clock.cc:648] HybridClock initialized: now 1755223735509161 us; error 32 us; skew 500 ppm
I20250815 02:08:55.510250 12257 webserver.cc:480] Webserver started at http://127.6.59.131:46337/ using document root <none> and password file <none>
I20250815 02:08:55.510440 12257 fs_manager.cc:362] Metadata directory not provided
I20250815 02:08:55.510500 12257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250815 02:08:55.511685 12257 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:55.512326 12272 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:55.512477 12257 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:55.512565 12257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal
uuid: "2a31cfc88db341028d33e8e4cd42ae0c"
format_stamp: "Formatted at 2025-08-15 02:08:54 on dist-test-slave-wq2q"
I20250815 02:08:55.512805 12257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250815 02:08:55.524045 12257 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250815 02:08:55.524276 12257 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250815 02:08:55.524384 12257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250815 02:08:55.524608 12257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250815 02:08:55.525041 12279 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250815 02:08:55.525926 12257 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250815 02:08:55.525971 12257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:55.526034 12257 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250815 02:08:55.526572 12257 ts_tablet_manager.cc:610] Registered 1 tablets
I20250815 02:08:55.526603 12257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250815 02:08:55.526655 12279 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Bootstrap starting.
I20250815 02:08:55.533758 12257 rpc_server.cc:307] RPC server started. Bound to: 127.6.59.131:33489
I20250815 02:08:55.533842 12386 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.59.131:33489 every 8 connection(s)
I20250815 02:08:55.534164 12257 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/info.pb
I20250815 02:08:55.537767 6382 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu as pid 12257
I20250815 02:08:55.538158 12279 log.cc:826] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Log is configured to *not* fsync() on all Append() calls
I20250815 02:08:55.539167 12387 heartbeater.cc:344] Connected to a master server at 127.6.59.190:42237
I20250815 02:08:55.539302 12387 heartbeater.cc:461] Registering TS with master...
I20250815 02:08:55.539494 12387 heartbeater.cc:507] Master 127.6.59.190:42237 requested a full tablet report, sending...
I20250815 02:08:55.539978 11931 ts_manager.cc:194] Registered new tserver with Master: 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131:33489)
I20250815 02:08:55.540472 11931 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.59.131:51333
I20250815 02:08:55.542338 6382 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250815 02:08:55.569828 12279 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=206 overwritten=0 applied=206 ignored=0} inserts{seen=10250 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250815 02:08:55.570214 12279 tablet_bootstrap.cc:492] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Bootstrap complete.
I20250815 02:08:55.571097 12279 ts_tablet_manager.cc:1397] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Time spent bootstrapping tablet: real 0.044s user 0.031s sys 0.012s
I20250815 02:08:55.571997 12279 raft_consensus.cc:357] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:55.572196 12279 raft_consensus.cc:738] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a31cfc88db341028d33e8e4cd42ae0c, State: Initialized, Role: FOLLOWER
I20250815 02:08:55.572299 12279 consensus_queue.cc:260] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 206, Last appended: 1.206, Last appended by leader: 206, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:55.572561 12279 ts_tablet_manager.cc:1428] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20250815 02:08:55.572672 12387 heartbeater.cc:499] Master 127.6.59.190:42237 was elected leader, sending a full tablet report...
I20250815 02:08:56.688000 12399 raft_consensus.cc:491] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250815 02:08:56.688122 12399 raft_consensus.cc:513] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:56.688429 12399 leader_election.cc:290] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131:33489), f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129:38827)
I20250815 02:08:56.691974 12341 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "e48639cfbf5f418e8de934daff9e77da" candidate_uuid: "93ce45ca80c8401e9cbb10f379397239" candidate_term: 2 candidate_status { last_received { term: 1 index: 206 } } ignore_live_leader: false dest_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" is_pre_election: true
I20250815 02:08:56.691973 12072 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "e48639cfbf5f418e8de934daff9e77da" candidate_uuid: "93ce45ca80c8401e9cbb10f379397239" candidate_term: 2 candidate_status { last_received { term: 1 index: 206 } } ignore_live_leader: false dest_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" is_pre_election: true
I20250815 02:08:56.692104 12341 raft_consensus.cc:2466] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93ce45ca80c8401e9cbb10f379397239 in term 1.
I20250815 02:08:56.692109 12072 raft_consensus.cc:2466] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93ce45ca80c8401e9cbb10f379397239 in term 1.
I20250815 02:08:56.692361 12139 leader_election.cc:304] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a31cfc88db341028d33e8e4cd42ae0c, 93ce45ca80c8401e9cbb10f379397239; no voters:
I20250815 02:08:56.692488 12399 raft_consensus.cc:2802] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250815 02:08:56.692553 12399 raft_consensus.cc:491] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250815 02:08:56.692589 12399 raft_consensus.cc:3058] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:56.693554 12399 raft_consensus.cc:513] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:56.693686 12399 leader_election.cc:290] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [CANDIDATE]: Term 2 election: Requested vote from peers 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131:33489), f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129:38827)
I20250815 02:08:56.693841 12072 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "e48639cfbf5f418e8de934daff9e77da" candidate_uuid: "93ce45ca80c8401e9cbb10f379397239" candidate_term: 2 candidate_status { last_received { term: 1 index: 206 } } ignore_live_leader: false dest_uuid: "f4354725d6514d32b8f4a8bebd6a9d94"
I20250815 02:08:56.693846 12341 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "e48639cfbf5f418e8de934daff9e77da" candidate_uuid: "93ce45ca80c8401e9cbb10f379397239" candidate_term: 2 candidate_status { last_received { term: 1 index: 206 } } ignore_live_leader: false dest_uuid: "2a31cfc88db341028d33e8e4cd42ae0c"
I20250815 02:08:56.693907 12341 raft_consensus.cc:3058] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:56.693912 12072 raft_consensus.cc:3058] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 1 FOLLOWER]: Advancing to term 2
I20250815 02:08:56.694782 12341 raft_consensus.cc:2466] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93ce45ca80c8401e9cbb10f379397239 in term 2.
I20250815 02:08:56.694828 12072 raft_consensus.cc:2466] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93ce45ca80c8401e9cbb10f379397239 in term 2.
I20250815 02:08:56.694952 12139 leader_election.cc:304] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a31cfc88db341028d33e8e4cd42ae0c, 93ce45ca80c8401e9cbb10f379397239; no voters:
I20250815 02:08:56.695045 12399 raft_consensus.cc:2802] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 2 FOLLOWER]: Leader election won for term 2
I20250815 02:08:56.695153 12399 raft_consensus.cc:695] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 2 LEADER]: Becoming Leader. State: Replica: 93ce45ca80c8401e9cbb10f379397239, State: Running, Role: LEADER
I20250815 02:08:56.695240 12399 consensus_queue.cc:237] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 206, Committed index: 206, Last appended: 1.206, Last appended by leader: 206, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:56.695917 11931 catalog_manager.cc:5582] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 reported cstate change: term changed from 0 to 2, leader changed from <none> to 93ce45ca80c8401e9cbb10f379397239 (127.6.59.130), VOTER 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131) added, VOTER 93ce45ca80c8401e9cbb10f379397239 (127.6.59.130) added, VOTER f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129) added. New cstate: current_term: 2 leader_uuid: "93ce45ca80c8401e9cbb10f379397239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } health_report { overall_health: UNKNOWN } } }
W20250815 02:08:57.058584 6382 scanner-internal.cc:458] Time spent opening tablet: real 1.510s user 0.001s sys 0.001s
I20250815 02:08:57.077709 12072 raft_consensus.cc:1273] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 2 FOLLOWER]: Refusing update from remote peer 93ce45ca80c8401e9cbb10f379397239: Log matching property violated. Preceding OpId in replica: term: 1 index: 206. Preceding OpId from leader: term: 2 index: 207. (index mismatch)
I20250815 02:08:57.078099 12399 consensus_queue.cc:1035] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 207, Last known committed idx: 206, Time since last communication: 0.000s
I20250815 02:08:57.080909 12341 raft_consensus.cc:1273] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 2 FOLLOWER]: Refusing update from remote peer 93ce45ca80c8401e9cbb10f379397239: Log matching property violated. Preceding OpId in replica: term: 1 index: 206. Preceding OpId from leader: term: 2 index: 207. (index mismatch)
I20250815 02:08:57.081362 12399 consensus_queue.cc:1035] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 207, Last known committed idx: 206, Time since last communication: 0.000s
I20250815 02:08:57.086257 12207 consensus_queue.cc:237] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 207, Committed index: 207, Last appended: 2.207, Last appended by leader: 206, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 208 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:57.086717 12072 raft_consensus.cc:1273] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 2 FOLLOWER]: Refusing update from remote peer 93ce45ca80c8401e9cbb10f379397239: Log matching property violated. Preceding OpId in replica: term: 2 index: 207. Preceding OpId from leader: term: 2 index: 208. (index mismatch)
I20250815 02:08:57.086926 12399 consensus_queue.cc:1035] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 208, Last known committed idx: 207, Time since last communication: 0.000s
I20250815 02:08:57.087661 12404 raft_consensus.cc:2953] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 2 LEADER]: Committing config change with OpId 2.208: config changed from index -1 to 208, VOTER 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131) evicted. New config: { opid_index: 208 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } } }
I20250815 02:08:57.089278 11918 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet e48639cfbf5f418e8de934daff9e77da with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250815 02:08:57.089607 11931 catalog_manager.cc:5582] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 reported cstate change: config changed from index -1 to 208, VOTER 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131) evicted. New cstate: current_term: 2 leader_uuid: "93ce45ca80c8401e9cbb10f379397239" committed_config { opid_index: 208 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } health_report { overall_health: HEALTHY } } }
I20250815 02:08:57.089565 12072 raft_consensus.cc:2953] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 2 FOLLOWER]: Committing config change with OpId 2.208: config changed from index -1 to 208, VOTER 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131) evicted. New config: { opid_index: 208 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } } }
I20250815 02:08:57.090974 12207 consensus_queue.cc:237] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 208, Committed index: 208, Last appended: 2.208, Last appended by leader: 206, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } }
I20250815 02:08:57.091236 12399 raft_consensus.cc:2953] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 2 LEADER]: Committing config change with OpId 2.209: config changed from index 208 to 209, VOTER f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129) evicted. New config: { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } }
I20250815 02:08:57.092590 11931 catalog_manager.cc:5582] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 reported cstate change: config changed from index 208 to 209, VOTER f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129) evicted. New cstate: current_term: 2 leader_uuid: "93ce45ca80c8401e9cbb10f379397239" committed_config { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } health_report { overall_health: HEALTHY } } }
I20250815 02:08:57.092757 11918 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet e48639cfbf5f418e8de934daff9e77da with cas_config_opid_index 208: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250815 02:08:57.094322 12321 tablet_service.cc:1515] Processing DeleteTablet for tablet e48639cfbf5f418e8de934daff9e77da with delete_type TABLET_DATA_TOMBSTONED (TS 2a31cfc88db341028d33e8e4cd42ae0c not found in new config with opid_index 208) from {username='slave'} at 127.0.0.1:35828
I20250815 02:08:57.094689 12449 tablet_replica.cc:331] stopping tablet replica
I20250815 02:08:57.094836 12449 raft_consensus.cc:2241] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250815 02:08:57.094954 12449 raft_consensus.cc:2270] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250815 02:08:57.095366 12449 ts_tablet_manager.cc:1905] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250815 02:08:57.096941 12449 ts_tablet_manager.cc:1918] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.207
I20250815 02:08:57.097070 12449 log.cc:1199] T e48639cfbf5f418e8de934daff9e77da P 2a31cfc88db341028d33e8e4cd42ae0c: Deleting WAL directory at /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/wal/wals/e48639cfbf5f418e8de934daff9e77da
I20250815 02:08:57.097592 11915 catalog_manager.cc:4928] TS 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131:33489): tablet e48639cfbf5f418e8de934daff9e77da (table pre_rebuild [id=6cc9238e07034c9f994018ff481ef09a]) successfully deleted
I20250815 02:08:57.099362 12052 tablet_service.cc:1515] Processing DeleteTablet for tablet e48639cfbf5f418e8de934daff9e77da with delete_type TABLET_DATA_TOMBSTONED (TS f4354725d6514d32b8f4a8bebd6a9d94 not found in new config with opid_index 209) from {username='slave'} at 127.0.0.1:37620
I20250815 02:08:57.099694 12450 tablet_replica.cc:331] stopping tablet replica
I20250815 02:08:57.099804 12450 raft_consensus.cc:2241] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250815 02:08:57.099825 12321 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:57.099895 12450 raft_consensus.cc:2270] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250815 02:08:57.100023 12187 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250815 02:08:57.100467 12450 ts_tablet_manager.cc:1905] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250815 02:08:57.101935 12450 ts_tablet_manager.cc:1918] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.208
I20250815 02:08:57.102068 12450 log.cc:1199] T e48639cfbf5f418e8de934daff9e77da P f4354725d6514d32b8f4a8bebd6a9d94: Deleting WAL directory at /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/wal/wals/e48639cfbf5f418e8de934daff9e77da
I20250815 02:08:57.102564 11918 catalog_manager.cc:4928] TS f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129:38827): tablet e48639cfbf5f418e8de934daff9e77da (table pre_rebuild [id=6cc9238e07034c9f994018ff481ef09a]) successfully deleted
I20250815 02:08:57.105041 12052 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
5ac491abad3145648f2f2eb3c30f220a | 127.6.59.190:42237 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.59.148:43943 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
2a31cfc88db341028d33e8e4cd42ae0c | 127.6.59.131:33489 | HEALTHY | <none> | 0 | 0
93ce45ca80c8401e9cbb10f379397239 | 127.6.59.130:45727 | HEALTHY | <none> | 1 | 0
f4354725d6514d32b8f4a8bebd6a9d94 | 127.6.59.129:38827 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.6.59.129 | experimental | 127.6.59.129:38827
local_ip_for_outbound_sockets | 127.6.59.130 | experimental | 127.6.59.130:45727
local_ip_for_outbound_sockets | 127.6.59.131 | experimental | 127.6.59.131:33489
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/info.pb | hidden | 127.6.59.129:38827
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/info.pb | hidden | 127.6.59.130:45727
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/info.pb | hidden | 127.6.59.131:33489
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.59.148:43943 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet e48639cfbf5f418e8de934daff9e77da of table 'pre_rebuild' is conflicted: 2 replicas' active configs disagree with the leader master's
93ce45ca80c8401e9cbb10f379397239 (127.6.59.130:45727): RUNNING [LEADER]
f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129:38827): RUNNING
All reported replicas are:
A = 93ce45ca80c8401e9cbb10f379397239
B = f4354725d6514d32b8f4a8bebd6a9d94
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+----------+--------------+--------------+------------
master | A* B | | | Yes
A | A* | 2 | 209 | Yes
B | A B | 2 | 208 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+--------------------+---------------+---------+------------+------------------+-------------
pre_rebuild | 1 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 2
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20250815 02:08:57.113178 6382 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20250815 02:08:57.286348 12321 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:57.286659 12052 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:57.287149 12187 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
5ac491abad3145648f2f2eb3c30f220a | 127.6.59.190:42237 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.59.148:43943 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
2a31cfc88db341028d33e8e4cd42ae0c | 127.6.59.131:33489 | HEALTHY | <none> | 0 | 0
93ce45ca80c8401e9cbb10f379397239 | 127.6.59.130:45727 | HEALTHY | <none> | 1 | 0
f4354725d6514d32b8f4a8bebd6a9d94 | 127.6.59.129:38827 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.6.59.129 | experimental | 127.6.59.129:38827
local_ip_for_outbound_sockets | 127.6.59.130 | experimental | 127.6.59.130:45727
local_ip_for_outbound_sockets | 127.6.59.131 | experimental | 127.6.59.131:33489
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/info.pb | hidden | 127.6.59.129:38827
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/info.pb | hidden | 127.6.59.130:45727
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/info.pb | hidden | 127.6.59.131:33489
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.59.148:43943 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
pre_rebuild | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 0
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20250815 02:08:57.296227 6382 log_verifier.cc:126] Checking tablet e48639cfbf5f418e8de934daff9e77da
I20250815 02:08:57.309239 6382 log_verifier.cc:177] Verified matching terms for 209 ops in tablet e48639cfbf5f418e8de934daff9e77da
I20250815 02:08:57.309743 11930 catalog_manager.cc:2482] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41906:
table { table_name: "pre_rebuild" } modify_external_catalogs: true
I20250815 02:08:57.309856 11930 catalog_manager.cc:2730] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41906:
table { table_name: "pre_rebuild" } modify_external_catalogs: true
I20250815 02:08:57.310633 11930 catalog_manager.cc:5869] T 00000000000000000000000000000000 P 5ac491abad3145648f2f2eb3c30f220a: Sending DeleteTablet for 1 replicas of tablet e48639cfbf5f418e8de934daff9e77da
I20250815 02:08:57.310927 12187 tablet_service.cc:1515] Processing DeleteTablet for tablet e48639cfbf5f418e8de934daff9e77da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-08-15 02:08:57 UTC) from {username='slave'} at 127.0.0.1:38454
I20250815 02:08:57.310964 6382 test_util.cc:276] Using random seed: -547545358
I20250815 02:08:57.311142 12474 tablet_replica.cc:331] stopping tablet replica
I20250815 02:08:57.311228 12474 raft_consensus.cc:2241] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 2 LEADER]: Raft consensus shutting down.
I20250815 02:08:57.311303 12474 raft_consensus.cc:2270] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250815 02:08:57.312431 12474 ts_tablet_manager.cc:1905] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250815 02:08:57.314304 12474 ts_tablet_manager.cc:1918] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.209
I20250815 02:08:57.314374 12474 log.cc:1199] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Deleting WAL directory at /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/wal/wals/e48639cfbf5f418e8de934daff9e77da
I20250815 02:08:57.314702 12474 ts_tablet_manager.cc:1939] T e48639cfbf5f418e8de934daff9e77da P 93ce45ca80c8401e9cbb10f379397239: Deleting consensus metadata
I20250815 02:08:57.315124 11918 catalog_manager.cc:4928] TS 93ce45ca80c8401e9cbb10f379397239 (127.6.59.130:45727): tablet e48639cfbf5f418e8de934daff9e77da (table pre_rebuild [id=6cc9238e07034c9f994018ff481ef09a]) successfully deleted
I20250815 02:08:57.316300 11930 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41974:
name: "post_rebuild"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250815 02:08:57.316591 11930 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table post_rebuild in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250815 02:08:57.318547 12321 tablet_service.cc:1468] Processing CreateTablet for tablet 7414a73f075d442db6fe32f96263fa62 (DEFAULT_TABLE table=post_rebuild [id=a19c714135ae4b519fcaddcd9d73e334]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:57.318611 12052 tablet_service.cc:1468] Processing CreateTablet for tablet 7414a73f075d442db6fe32f96263fa62 (DEFAULT_TABLE table=post_rebuild [id=a19c714135ae4b519fcaddcd9d73e334]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:57.318745 12321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7414a73f075d442db6fe32f96263fa62. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:57.318758 12052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7414a73f075d442db6fe32f96263fa62. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:57.320127 12480 tablet_bootstrap.cc:492] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c: Bootstrap starting.
I20250815 02:08:57.320209 12187 tablet_service.cc:1468] Processing CreateTablet for tablet 7414a73f075d442db6fe32f96263fa62 (DEFAULT_TABLE table=post_rebuild [id=a19c714135ae4b519fcaddcd9d73e334]), partition=RANGE (key) PARTITION UNBOUNDED
I20250815 02:08:57.320359 12187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7414a73f075d442db6fe32f96263fa62. 1 dirs total, 0 dirs full, 0 dirs failed
I20250815 02:08:57.320508 12480 tablet_bootstrap.cc:654] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:57.320851 12481 tablet_bootstrap.cc:492] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94: Bootstrap starting.
I20250815 02:08:57.321072 12480 tablet_bootstrap.cc:492] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c: No bootstrap required, opened a new log
I20250815 02:08:57.321128 12480 ts_tablet_manager.cc:1397] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:57.321147 12481 tablet_bootstrap.cc:654] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:57.321274 12480 raft_consensus.cc:357] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:57.321338 12480 raft_consensus.cc:383] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:57.321364 12480 raft_consensus.cc:738] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a31cfc88db341028d33e8e4cd42ae0c, State: Initialized, Role: FOLLOWER
I20250815 02:08:57.321436 12480 consensus_queue.cc:260] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:57.321565 12480 ts_tablet_manager.cc:1428] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250815 02:08:57.321811 12481 tablet_bootstrap.cc:492] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94: No bootstrap required, opened a new log
I20250815 02:08:57.321904 12481 ts_tablet_manager.cc:1397] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:57.322091 12481 raft_consensus.cc:357] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:57.322160 12481 raft_consensus.cc:383] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:57.322198 12481 raft_consensus.cc:738] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4354725d6514d32b8f4a8bebd6a9d94, State: Initialized, Role: FOLLOWER
I20250815 02:08:57.322263 12481 consensus_queue.cc:260] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:57.322368 12481 ts_tablet_manager.cc:1428] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:57.322471 12484 tablet_bootstrap.cc:492] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239: Bootstrap starting.
I20250815 02:08:57.322825 12484 tablet_bootstrap.cc:654] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239: Neither blocks nor log segments found. Creating new log.
I20250815 02:08:57.323408 12484 tablet_bootstrap.cc:492] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239: No bootstrap required, opened a new log
I20250815 02:08:57.323454 12484 ts_tablet_manager.cc:1397] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250815 02:08:57.323532 12484 raft_consensus.cc:357] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:57.323576 12484 raft_consensus.cc:383] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250815 02:08:57.323589 12484 raft_consensus.cc:738] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ce45ca80c8401e9cbb10f379397239, State: Initialized, Role: FOLLOWER
I20250815 02:08:57.323624 12484 consensus_queue.cc:260] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:57.323719 12484 ts_tablet_manager.cc:1428] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250815 02:08:57.410912 12404 raft_consensus.cc:491] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250815 02:08:57.411016 12404 raft_consensus.cc:513] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:57.411162 12404 leader_election.cc:290] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131:33489), f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129:38827)
I20250815 02:08:57.411386 12072 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7414a73f075d442db6fe32f96263fa62" candidate_uuid: "93ce45ca80c8401e9cbb10f379397239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" is_pre_election: true
I20250815 02:08:57.411396 12341 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7414a73f075d442db6fe32f96263fa62" candidate_uuid: "93ce45ca80c8401e9cbb10f379397239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" is_pre_election: true
I20250815 02:08:57.411478 12341 raft_consensus.cc:2466] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93ce45ca80c8401e9cbb10f379397239 in term 0.
I20250815 02:08:57.411478 12072 raft_consensus.cc:2466] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93ce45ca80c8401e9cbb10f379397239 in term 0.
I20250815 02:08:57.411652 12142 leader_election.cc:304] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93ce45ca80c8401e9cbb10f379397239, f4354725d6514d32b8f4a8bebd6a9d94; no voters:
I20250815 02:08:57.411746 12404 raft_consensus.cc:2802] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250815 02:08:57.411798 12404 raft_consensus.cc:491] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250815 02:08:57.411834 12404 raft_consensus.cc:3058] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:57.412427 12404 raft_consensus.cc:513] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:57.412576 12404 leader_election.cc:290] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [CANDIDATE]: Term 1 election: Requested vote from peers 2a31cfc88db341028d33e8e4cd42ae0c (127.6.59.131:33489), f4354725d6514d32b8f4a8bebd6a9d94 (127.6.59.129:38827)
W20250815 02:08:57.412670 12254 tablet.cc:2378] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250815 02:08:57.412745 12341 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7414a73f075d442db6fe32f96263fa62" candidate_uuid: "93ce45ca80c8401e9cbb10f379397239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a31cfc88db341028d33e8e4cd42ae0c"
I20250815 02:08:57.412810 12341 raft_consensus.cc:3058] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:57.412807 12072 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7414a73f075d442db6fe32f96263fa62" candidate_uuid: "93ce45ca80c8401e9cbb10f379397239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4354725d6514d32b8f4a8bebd6a9d94"
I20250815 02:08:57.412873 12072 raft_consensus.cc:3058] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94 [term 0 FOLLOWER]: Advancing to term 1
I20250815 02:08:57.413371 12341 raft_consensus.cc:2466] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93ce45ca80c8401e9cbb10f379397239 in term 1.
I20250815 02:08:57.413388 12072 raft_consensus.cc:2466] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93ce45ca80c8401e9cbb10f379397239 in term 1.
I20250815 02:08:57.413547 12142 leader_election.cc:304] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2a31cfc88db341028d33e8e4cd42ae0c, 93ce45ca80c8401e9cbb10f379397239, f4354725d6514d32b8f4a8bebd6a9d94; no voters:
I20250815 02:08:57.413637 12404 raft_consensus.cc:2802] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 1 FOLLOWER]: Leader election won for term 1
I20250815 02:08:57.413686 12404 raft_consensus.cc:695] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [term 1 LEADER]: Becoming Leader. State: Replica: 93ce45ca80c8401e9cbb10f379397239, State: Running, Role: LEADER
I20250815 02:08:57.413746 12404 consensus_queue.cc:237] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } } peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } }
I20250815 02:08:57.414275 11931 catalog_manager.cc:5582] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93ce45ca80c8401e9cbb10f379397239 (127.6.59.130). New cstate: current_term: 1 leader_uuid: "93ce45ca80c8401e9cbb10f379397239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93ce45ca80c8401e9cbb10f379397239" member_type: VOTER last_known_addr { host: "127.6.59.130" port: 45727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 } health_report { overall_health: UNKNOWN } } }
I20250815 02:08:57.448499 12341 raft_consensus.cc:1273] T 7414a73f075d442db6fe32f96263fa62 P 2a31cfc88db341028d33e8e4cd42ae0c [term 1 FOLLOWER]: Refusing update from remote peer 93ce45ca80c8401e9cbb10f379397239: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250815 02:08:57.448518 12072 raft_consensus.cc:1273] T 7414a73f075d442db6fe32f96263fa62 P f4354725d6514d32b8f4a8bebd6a9d94 [term 1 FOLLOWER]: Refusing update from remote peer 93ce45ca80c8401e9cbb10f379397239: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250815 02:08:57.448745 12399 consensus_queue.cc:1035] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4354725d6514d32b8f4a8bebd6a9d94" member_type: VOTER last_known_addr { host: "127.6.59.129" port: 38827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:57.448933 12399 consensus_queue.cc:1035] T 7414a73f075d442db6fe32f96263fa62 P 93ce45ca80c8401e9cbb10f379397239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a31cfc88db341028d33e8e4cd42ae0c" member_type: VOTER last_known_addr { host: "127.6.59.131" port: 33489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250815 02:08:57.451066 12428 mvcc.cc:204] Tried to move back new op lower bound from 7189396428589121536 to 7189396428447027200. Current Snapshot: MvccSnapshot[applied={T|T < 7189396428587266048}]
I20250815 02:08:57.451228 12429 mvcc.cc:204] Tried to move back new op lower bound from 7189396428589121536 to 7189396428447027200. Current Snapshot: MvccSnapshot[applied={T|T < 7189396428589121536 or (T in {7189396428589121536})}]
I20250815 02:08:57.452116 12433 mvcc.cc:204] Tried to move back new op lower bound from 7189396428589121536 to 7189396428447027200. Current Snapshot: MvccSnapshot[applied={T|T < 7189396428587266048}]
I20250815 02:08:57.622741 12321 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250815 02:08:57.623982 12187 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250815 02:08:57.626680 12052 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
5ac491abad3145648f2f2eb3c30f220a | 127.6.59.190:42237 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.59.148:43943 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
2a31cfc88db341028d33e8e4cd42ae0c | 127.6.59.131:33489 | HEALTHY | <none> | 0 | 0
93ce45ca80c8401e9cbb10f379397239 | 127.6.59.130:45727 | HEALTHY | <none> | 1 | 0
f4354725d6514d32b8f4a8bebd6a9d94 | 127.6.59.129:38827 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.6.59.129 | experimental | 127.6.59.129:38827
local_ip_for_outbound_sockets | 127.6.59.130 | experimental | 127.6.59.130:45727
local_ip_for_outbound_sockets | 127.6.59.131 | experimental | 127.6.59.131:33489
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-0/data/info.pb | hidden | 127.6.59.129:38827
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-1/data/info.pb | hidden | 127.6.59.130:45727
server_dump_info_path | /tmp/dist-test-taskpPPZ_f/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1755223715463109-6382-0/minicluster-data/ts-2/data/info.pb | hidden | 127.6.59.131:33489
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.59.148:43943 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
--------------+----+---------+---------------+---------+------------+------------------+-------------
post_rebuild | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20250815 02:08:57.633853 6382 log_verifier.cc:126] Checking tablet 7414a73f075d442db6fe32f96263fa62
I20250815 02:08:57.670245 6382 log_verifier.cc:177] Verified matching terms for 214 ops in tablet 7414a73f075d442db6fe32f96263fa62
I20250815 02:08:57.670331 6382 log_verifier.cc:126] Checking tablet e48639cfbf5f418e8de934daff9e77da
I20250815 02:08:57.670398 6382 log_verifier.cc:177] Verified matching terms for 0 ops in tablet e48639cfbf5f418e8de934daff9e77da
I20250815 02:08:57.672246 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 11989
I20250815 02:08:57.677408 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 12123
I20250815 02:08:57.682721 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 12257
I20250815 02:08:57.687345 6382 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskpPPZ_f/build/release/bin/kudu with pid 11900
2025-08-15T02:08:57Z chronyd exiting
[ OK ] IsSecure/SecureClusterAdminCliParamTest.TestRebuildMaster/0 (4157 ms)
[----------] 1 test from IsSecure/SecureClusterAdminCliParamTest (4157 ms total)
[----------] Global test environment tear-down
[==========] 9 tests from 5 test suites ran. (22241 ms total)
[ PASSED ] 8 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] AdminCliTest.TestRebuildTables
1 FAILED TEST
I20250815 02:08:57.709815 6382 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/client/meta_cache.cc:302: suppressed but not reported on 4 messages since previous log ~9 seconds ago