Note: This is test shard 6 of 8.
[==========] Running 9 tests from 5 test suites.
[----------] Global test environment set-up.
[----------] 5 tests from AdminCliTest
[ RUN ] AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251024 08:15:00.612769 32207 test_util.cc:276] Using random seed: 626967179
I20251024 08:15:00.618491 32207 ts_itest-base.cc:115] Starting cluster with:
I20251024 08:15:00.618600 32207 ts_itest-base.cc:116] --------------
I20251024 08:15:00.618613 32207 ts_itest-base.cc:117] 4 tablet servers
I20251024 08:15:00.618628 32207 ts_itest-base.cc:118] 3 replicas per TS
I20251024 08:15:00.618639 32207 ts_itest-base.cc:119] --------------
2025-10-24T08:15:00Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-24T08:15:00Z Disabled control of system clock
I20251024 08:15:00.628201 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:34799
--webserver_interface=127.31.115.254
--webserver_port=0
--builtin_ntp_servers=127.31.115.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:34799 with env {}
W20251024 08:15:00.713253 32219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:00.713470 32219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:00.713492 32219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:00.715173 32219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:00.715243 32219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:00.715258 32219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:00.715272 32219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:00.717125 32219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:34799
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:34799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.32219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:00.717410 32219 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:00.717723 32219 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:00.720930 32225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:00.720909 32227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:00.720950 32219 server_base.cc:1047] running on GCE node
W20251024 08:15:00.720914 32224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:00.721395 32219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:00.721693 32219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:00.722918 32219 hybrid_clock.cc:648] HybridClock initialized: now 1761293700722891 us; error 42 us; skew 500 ppm
I20251024 08:15:00.724514 32219 webserver.cc:492] Webserver started at http://127.31.115.254:44173/ using document root <none> and password file <none>
I20251024 08:15:00.724761 32219 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:00.724813 32219 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:00.724951 32219 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:00.725917 32219 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "c62030364a3841969b2af8dc1ed94f73"
format_stamp: "Formatted at 2025-10-24 08:15:00 on dist-test-slave-c0j3"
I20251024 08:15:00.726274 32219 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "c62030364a3841969b2af8dc1ed94f73"
format_stamp: "Formatted at 2025-10-24 08:15:00 on dist-test-slave-c0j3"
I20251024 08:15:00.727730 32219 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20251024 08:15:00.728631 32233 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:00.728825 32219 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:00.728919 32219 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
uuid: "c62030364a3841969b2af8dc1ed94f73"
format_stamp: "Formatted at 2025-10-24 08:15:00 on dist-test-slave-c0j3"
I20251024 08:15:00.728997 32219 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:00.748579 32219 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:00.748925 32219 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:00.749087 32219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:00.753410 32219 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:34799
I20251024 08:15:00.753473 32285 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:34799 every 8 connection(s)
I20251024 08:15:00.753820 32219 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20251024 08:15:00.754585 32286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:00.757220 32286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73: Bootstrap starting.
I20251024 08:15:00.757889 32286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:00.758149 32286 log.cc:826] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:00.758960 32286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73: No bootstrap required, opened a new log
I20251024 08:15:00.760301 32286 raft_consensus.cc:359] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } }
I20251024 08:15:00.760466 32286 raft_consensus.cc:385] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:00.760488 32286 raft_consensus.cc:740] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62030364a3841969b2af8dc1ed94f73, State: Initialized, Role: FOLLOWER
I20251024 08:15:00.760608 32286 consensus_queue.cc:260] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } }
I20251024 08:15:00.760690 32286 raft_consensus.cc:399] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:00.760739 32286 raft_consensus.cc:493] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:00.760779 32286 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:00.761390 32286 raft_consensus.cc:515] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } }
I20251024 08:15:00.761507 32286 leader_election.cc:304] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c62030364a3841969b2af8dc1ed94f73; no voters:
I20251024 08:15:00.761678 32286 leader_election.cc:290] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:00.761775 32289 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:00.761969 32289 raft_consensus.cc:697] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 1 LEADER]: Becoming Leader. State: Replica: c62030364a3841969b2af8dc1ed94f73, State: Running, Role: LEADER
I20251024 08:15:00.761973 32286 sys_catalog.cc:565] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:00.762138 32289 consensus_queue.cc:237] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } }
I20251024 08:15:00.762655 32291 sys_catalog.cc:455] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c62030364a3841969b2af8dc1ed94f73. Latest consensus state: current_term: 1 leader_uuid: "c62030364a3841969b2af8dc1ed94f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } } }
I20251024 08:15:00.762688 32290 sys_catalog.cc:455] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c62030364a3841969b2af8dc1ed94f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } } }
I20251024 08:15:00.762874 32290 sys_catalog.cc:458] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:00.762858 32291 sys_catalog.cc:458] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:00.763243 32298 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:00.763836 32298 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:00.764200 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 32219
I20251024 08:15:00.764325 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance
I20251024 08:15:00.766063 32298 catalog_manager.cc:1357] Generated new cluster ID: 610f182c481144acac15c7e1d71b2551
I20251024 08:15:00.766144 32298 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:00.780351 32298 catalog_manager.cc:1380] Generated new certificate authority record
I20251024 08:15:00.780972 32298 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:00.788113 32298 catalog_manager.cc:6022] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73: Generated new TSK 0
I20251024 08:15:00.788358 32298 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251024 08:15:00.793164 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.193:0
--local_ip_for_outbound_sockets=127.31.115.193
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:34799
--builtin_ntp_servers=127.31.115.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20251024 08:15:00.880193 32311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:00.880426 32311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:00.880450 32311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:00.882182 32311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:00.882287 32311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.193
I20251024 08:15:00.884208 32311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:34799
--never_fsync=true
--heap_profile_path=/tmp/kudu.32311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.193
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:00.884496 32311 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:00.884757 32311 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:00.887727 32319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:00.887729 32317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:00.887925 32311 server_base.cc:1047] running on GCE node
W20251024 08:15:00.887727 32316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:00.888325 32311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:00.888581 32311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:00.889734 32311 hybrid_clock.cc:648] HybridClock initialized: now 1761293700889718 us; error 38 us; skew 500 ppm
I20251024 08:15:00.891304 32311 webserver.cc:492] Webserver started at http://127.31.115.193:35077/ using document root <none> and password file <none>
I20251024 08:15:00.891538 32311 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:00.891597 32311 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:00.891738 32311 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:00.892765 32311 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "1cbadc975b064d5e88f5c479a6ce1ed9"
format_stamp: "Formatted at 2025-10-24 08:15:00 on dist-test-slave-c0j3"
I20251024 08:15:00.893128 32311 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "1cbadc975b064d5e88f5c479a6ce1ed9"
format_stamp: "Formatted at 2025-10-24 08:15:00 on dist-test-slave-c0j3"
I20251024 08:15:00.894587 32311 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20251024 08:15:00.895534 32325 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:00.895776 32311 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:00.895861 32311 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "1cbadc975b064d5e88f5c479a6ce1ed9"
format_stamp: "Formatted at 2025-10-24 08:15:00 on dist-test-slave-c0j3"
I20251024 08:15:00.895943 32311 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:00.919778 32311 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:00.920059 32311 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:00.920157 32311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:00.920413 32311 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:00.920773 32311 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:00.920807 32311 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:00.920833 32311 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:00.920861 32311 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:00.928941 32311 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.193:43133
I20251024 08:15:00.929086 32438 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.193:43133 every 8 connection(s)
I20251024 08:15:00.929488 32311 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20251024 08:15:00.935979 32439 heartbeater.cc:344] Connected to a master server at 127.31.115.254:34799
I20251024 08:15:00.936123 32439 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:00.936383 32439 heartbeater.cc:507] Master 127.31.115.254:34799 requested a full tablet report, sending...
I20251024 08:15:00.936946 32250 ts_manager.cc:194] Registered new tserver with Master: 1cbadc975b064d5e88f5c479a6ce1ed9 (127.31.115.193:43133)
I20251024 08:15:00.937713 32250 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.193:39437
I20251024 08:15:00.939141 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 32311
I20251024 08:15:00.939234 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20251024 08:15:00.940503 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.194:0
--local_ip_for_outbound_sockets=127.31.115.194
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:34799
--builtin_ntp_servers=127.31.115.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20251024 08:15:01.025611 32442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:01.025830 32442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:01.025853 32442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:01.027478 32442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:01.027576 32442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.194
I20251024 08:15:01.029354 32442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:34799
--never_fsync=true
--heap_profile_path=/tmp/kudu.32442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.194
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:01.029619 32442 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:01.029852 32442 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:01.032716 32450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:01.032739 32448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:01.032802 32447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:01.033308 32442 server_base.cc:1047] running on GCE node
I20251024 08:15:01.033561 32442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:01.033851 32442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:01.035019 32442 hybrid_clock.cc:648] HybridClock initialized: now 1761293701034982 us; error 53 us; skew 500 ppm
I20251024 08:15:01.036556 32442 webserver.cc:492] Webserver started at http://127.31.115.194:33523/ using document root <none> and password file <none>
I20251024 08:15:01.036785 32442 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:01.036835 32442 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:01.036955 32442 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:01.037885 32442 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "cb057c9691e74d15a4e558a7db6f302f"
format_stamp: "Formatted at 2025-10-24 08:15:01 on dist-test-slave-c0j3"
I20251024 08:15:01.038208 32442 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "cb057c9691e74d15a4e558a7db6f302f"
format_stamp: "Formatted at 2025-10-24 08:15:01 on dist-test-slave-c0j3"
I20251024 08:15:01.039672 32442 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20251024 08:15:01.040552 32456 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:01.040745 32442 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:01.040834 32442 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "cb057c9691e74d15a4e558a7db6f302f"
format_stamp: "Formatted at 2025-10-24 08:15:01 on dist-test-slave-c0j3"
I20251024 08:15:01.040905 32442 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:01.052325 32442 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:01.052625 32442 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:01.052742 32442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:01.052982 32442 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:01.053288 32442 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:01.053323 32442 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:01.053360 32442 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:01.053388 32442 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:01.059552 32442 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.194:39567
I20251024 08:15:01.059623 32570 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.194:39567 every 8 connection(s)
I20251024 08:15:01.059978 32442 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20251024 08:15:01.066275 32571 heartbeater.cc:344] Connected to a master server at 127.31.115.254:34799
I20251024 08:15:01.066443 32571 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:01.066560 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 32442
I20251024 08:15:01.066668 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20251024 08:15:01.066823 32571 heartbeater.cc:507] Master 127.31.115.254:34799 requested a full tablet report, sending...
I20251024 08:15:01.067380 32249 ts_manager.cc:194] Registered new tserver with Master: cb057c9691e74d15a4e558a7db6f302f (127.31.115.194:39567)
I20251024 08:15:01.067879 32249 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.194:48821
I20251024 08:15:01.068008 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.195:0
--local_ip_for_outbound_sockets=127.31.115.195
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:34799
--builtin_ntp_servers=127.31.115.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20251024 08:15:01.155040 32574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:01.155294 32574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:01.155321 32574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:01.157013 32574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:01.157123 32574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.195
I20251024 08:15:01.159054 32574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:34799
--never_fsync=true
--heap_profile_path=/tmp/kudu.32574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.195
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:01.159350 32574 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:01.159646 32574 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:01.162761 32580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:01.162792 32582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:01.162761 32579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:01.163725 32574 server_base.cc:1047] running on GCE node
I20251024 08:15:01.163930 32574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:01.164206 32574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:01.165396 32574 hybrid_clock.cc:648] HybridClock initialized: now 1761293701165375 us; error 45 us; skew 500 ppm
I20251024 08:15:01.167023 32574 webserver.cc:492] Webserver started at http://127.31.115.195:44971/ using document root <none> and password file <none>
I20251024 08:15:01.167318 32574 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:01.167388 32574 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:01.167531 32574 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:01.168529 32574 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "8fa56def48344ca49f9a6520dcd0033d"
format_stamp: "Formatted at 2025-10-24 08:15:01 on dist-test-slave-c0j3"
I20251024 08:15:01.168895 32574 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "8fa56def48344ca49f9a6520dcd0033d"
format_stamp: "Formatted at 2025-10-24 08:15:01 on dist-test-slave-c0j3"
I20251024 08:15:01.170634 32574 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20251024 08:15:01.171787 32588 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:01.172099 32574 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:01.172233 32574 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "8fa56def48344ca49f9a6520dcd0033d"
format_stamp: "Formatted at 2025-10-24 08:15:01 on dist-test-slave-c0j3"
I20251024 08:15:01.172317 32574 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:01.196882 32574 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:01.197209 32574 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:01.197350 32574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:01.197597 32574 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:01.197978 32574 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:01.198020 32574 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:01.198061 32574 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:01.198099 32574 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:01.205272 32574 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.195:33929
I20251024 08:15:01.205353 32702 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.195:33929 every 8 connection(s)
I20251024 08:15:01.205724 32574 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20251024 08:15:01.211026 32703 heartbeater.cc:344] Connected to a master server at 127.31.115.254:34799
I20251024 08:15:01.211161 32703 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:01.211444 32703 heartbeater.cc:507] Master 127.31.115.254:34799 requested a full tablet report, sending...
I20251024 08:15:01.211866 32249 ts_manager.cc:194] Registered new tserver with Master: 8fa56def48344ca49f9a6520dcd0033d (127.31.115.195:33929)
I20251024 08:15:01.212232 32249 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.195:34483
I20251024 08:15:01.214920 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 32574
I20251024 08:15:01.215045 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20251024 08:15:01.216694 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.196:0
--local_ip_for_outbound_sockets=127.31.115.196
--webserver_interface=127.31.115.196
--webserver_port=0
--tserver_master_addrs=127.31.115.254:34799
--builtin_ntp_servers=127.31.115.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20251024 08:15:01.308094 32706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:01.308327 32706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:01.308349 32706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:01.310037 32706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:01.310128 32706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.196
I20251024 08:15:01.312065 32706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.31.115.196
--webserver_port=0
--tserver_master_addrs=127.31.115.254:34799
--never_fsync=true
--heap_profile_path=/tmp/kudu.32706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.196
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:01.312387 32706 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:01.312659 32706 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251024 08:15:01.315726 32706 server_base.cc:1047] running on GCE node
W20251024 08:15:01.315703 32712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:01.315712 32711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:01.315690 32714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:01.316075 32706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:01.316318 32706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:01.317498 32706 hybrid_clock.cc:648] HybridClock initialized: now 1761293701317478 us; error 37 us; skew 500 ppm
I20251024 08:15:01.318996 32706 webserver.cc:492] Webserver started at http://127.31.115.196:33083/ using document root <none> and password file <none>
I20251024 08:15:01.319202 32706 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:01.319257 32706 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:01.319363 32706 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:01.320322 32706 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "e617e2c40edf4e3bac1529df22783efa"
format_stamp: "Formatted at 2025-10-24 08:15:01 on dist-test-slave-c0j3"
I20251024 08:15:01.320639 32706 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "e617e2c40edf4e3bac1529df22783efa"
format_stamp: "Formatted at 2025-10-24 08:15:01 on dist-test-slave-c0j3"
I20251024 08:15:01.321944 32706 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20251024 08:15:01.322928 32720 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:01.323127 32706 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:01.323220 32706 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "e617e2c40edf4e3bac1529df22783efa"
format_stamp: "Formatted at 2025-10-24 08:15:01 on dist-test-slave-c0j3"
I20251024 08:15:01.323313 32706 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:01.335659 32706 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:01.335992 32706 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:01.336117 32706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:01.336374 32706 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:01.336733 32706 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:01.336771 32706 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:01.336810 32706 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:01.336843 32706 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:01.343171 32706 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.196:42933
I20251024 08:15:01.343255 365 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.196:42933 every 8 connection(s)
I20251024 08:15:01.343595 32706 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20251024 08:15:01.349448 366 heartbeater.cc:344] Connected to a master server at 127.31.115.254:34799
I20251024 08:15:01.349603 366 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:01.349823 366 heartbeater.cc:507] Master 127.31.115.254:34799 requested a full tablet report, sending...
I20251024 08:15:01.350339 32249 ts_manager.cc:194] Registered new tserver with Master: e617e2c40edf4e3bac1529df22783efa (127.31.115.196:42933)
I20251024 08:15:01.350800 32249 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.196:58351
I20251024 08:15:01.353008 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 32706
I20251024 08:15:01.353111 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20251024 08:15:01.354954 32207 external_mini_cluster.cc:949] 4 TS(s) registered with all masters
I20251024 08:15:01.362435 32249 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60858:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20251024 08:15:01.371423 32636 tablet_service.cc:1505] Processing CreateTablet for tablet 8ae9089ad39344bc86eb7800e400de64 (DEFAULT_TABLE table=TestTable [id=5fa9fdc0a1314328b3698870c7c60564]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:01.371609 300 tablet_service.cc:1505] Processing CreateTablet for tablet 8ae9089ad39344bc86eb7800e400de64 (DEFAULT_TABLE table=TestTable [id=5fa9fdc0a1314328b3698870c7c60564]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:01.371829 32636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae9089ad39344bc86eb7800e400de64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:01.371943 300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae9089ad39344bc86eb7800e400de64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:01.372174 32373 tablet_service.cc:1505] Processing CreateTablet for tablet 8ae9089ad39344bc86eb7800e400de64 (DEFAULT_TABLE table=TestTable [id=5fa9fdc0a1314328b3698870c7c60564]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:01.372489 32373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae9089ad39344bc86eb7800e400de64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:01.374449 383 tablet_bootstrap.cc:492] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa: Bootstrap starting.
I20251024 08:15:01.375072 384 tablet_bootstrap.cc:492] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d: Bootstrap starting.
I20251024 08:15:01.375399 383 tablet_bootstrap.cc:654] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:01.375766 384 tablet_bootstrap.cc:654] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:01.375777 385 tablet_bootstrap.cc:492] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9: Bootstrap starting.
I20251024 08:15:01.375723 383 log.cc:826] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:01.376081 384 log.cc:826] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:01.376585 385 tablet_bootstrap.cc:654] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:01.376616 383 tablet_bootstrap.cc:492] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa: No bootstrap required, opened a new log
I20251024 08:15:01.376674 383 ts_tablet_manager.cc:1403] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:01.376730 384 tablet_bootstrap.cc:492] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d: No bootstrap required, opened a new log
I20251024 08:15:01.376827 384 ts_tablet_manager.cc:1403] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251024 08:15:01.376955 385 log.cc:826] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:01.377724 385 tablet_bootstrap.cc:492] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9: No bootstrap required, opened a new log
I20251024 08:15:01.377792 385 ts_tablet_manager.cc:1403] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:01.378409 384 raft_consensus.cc:359] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }
I20251024 08:15:01.378602 384 raft_consensus.cc:385] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:01.378645 384 raft_consensus.cc:740] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fa56def48344ca49f9a6520dcd0033d, State: Initialized, Role: FOLLOWER
I20251024 08:15:01.378607 383 raft_consensus.cc:359] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }
I20251024 08:15:01.378746 383 raft_consensus.cc:385] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:01.378785 383 raft_consensus.cc:740] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e617e2c40edf4e3bac1529df22783efa, State: Initialized, Role: FOLLOWER
I20251024 08:15:01.378774 384 consensus_queue.cc:260] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }
I20251024 08:15:01.378885 383 consensus_queue.cc:260] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }
I20251024 08:15:01.379051 384 ts_tablet_manager.cc:1434] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251024 08:15:01.379159 383 ts_tablet_manager.cc:1434] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251024 08:15:01.379189 32703 heartbeater.cc:499] Master 127.31.115.254:34799 was elected leader, sending a full tablet report...
I20251024 08:15:01.379451 366 heartbeater.cc:499] Master 127.31.115.254:34799 was elected leader, sending a full tablet report...
I20251024 08:15:01.379570 385 raft_consensus.cc:359] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }
I20251024 08:15:01.379724 385 raft_consensus.cc:385] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:01.379761 385 raft_consensus.cc:740] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cbadc975b064d5e88f5c479a6ce1ed9, State: Initialized, Role: FOLLOWER
I20251024 08:15:01.379863 385 consensus_queue.cc:260] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }
I20251024 08:15:01.380148 385 ts_tablet_manager.cc:1434] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251024 08:15:01.380290 32439 heartbeater.cc:499] Master 127.31.115.254:34799 was elected leader, sending a full tablet report...
I20251024 08:15:01.395143 391 raft_consensus.cc:493] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251024 08:15:01.395282 391 raft_consensus.cc:515] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }
I20251024 08:15:01.395632 391 leader_election.cc:290] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e617e2c40edf4e3bac1529df22783efa (127.31.115.196:42933), 8fa56def48344ca49f9a6520dcd0033d (127.31.115.195:33929)
I20251024 08:15:01.399494 320 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ae9089ad39344bc86eb7800e400de64" candidate_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e617e2c40edf4e3bac1529df22783efa" is_pre_election: true
I20251024 08:15:01.399711 320 raft_consensus.cc:2468] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cbadc975b064d5e88f5c479a6ce1ed9 in term 0.
I20251024 08:15:01.399803 32657 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ae9089ad39344bc86eb7800e400de64" candidate_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fa56def48344ca49f9a6520dcd0033d" is_pre_election: true
I20251024 08:15:01.399976 32657 raft_consensus.cc:2468] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cbadc975b064d5e88f5c479a6ce1ed9 in term 0.
I20251024 08:15:01.399977 32328 leader_election.cc:304] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cbadc975b064d5e88f5c479a6ce1ed9, e617e2c40edf4e3bac1529df22783efa; no voters:
I20251024 08:15:01.400110 391 raft_consensus.cc:2804] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251024 08:15:01.400183 391 raft_consensus.cc:493] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251024 08:15:01.400218 391 raft_consensus.cc:3060] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:01.401006 391 raft_consensus.cc:515] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }
I20251024 08:15:01.401168 391 leader_election.cc:290] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [CANDIDATE]: Term 1 election: Requested vote from peers e617e2c40edf4e3bac1529df22783efa (127.31.115.196:42933), 8fa56def48344ca49f9a6520dcd0033d (127.31.115.195:33929)
I20251024 08:15:01.401340 320 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ae9089ad39344bc86eb7800e400de64" candidate_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e617e2c40edf4e3bac1529df22783efa"
I20251024 08:15:01.401391 32657 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ae9089ad39344bc86eb7800e400de64" candidate_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fa56def48344ca49f9a6520dcd0033d"
I20251024 08:15:01.401435 320 raft_consensus.cc:3060] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:01.401458 32657 raft_consensus.cc:3060] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:01.402102 32657 raft_consensus.cc:2468] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cbadc975b064d5e88f5c479a6ce1ed9 in term 1.
I20251024 08:15:01.402124 320 raft_consensus.cc:2468] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cbadc975b064d5e88f5c479a6ce1ed9 in term 1.
I20251024 08:15:01.402303 32328 leader_election.cc:304] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cbadc975b064d5e88f5c479a6ce1ed9, e617e2c40edf4e3bac1529df22783efa; no voters:
I20251024 08:15:01.402436 391 raft_consensus.cc:2804] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:01.402648 391 raft_consensus.cc:697] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [term 1 LEADER]: Becoming Leader. State: Replica: 1cbadc975b064d5e88f5c479a6ce1ed9, State: Running, Role: LEADER
I20251024 08:15:01.402772 391 consensus_queue.cc:237] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }
I20251024 08:15:01.403626 32250 catalog_manager.cc:5649] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cbadc975b064d5e88f5c479a6ce1ed9 (127.31.115.193). New cstate: current_term: 1 leader_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:01.411582 32207 external_mini_cluster.cc:949] 4 TS(s) registered with all masters
I20251024 08:15:01.412425 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 1cbadc975b064d5e88f5c479a6ce1ed9 to finish bootstrapping
I20251024 08:15:01.416788 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 8fa56def48344ca49f9a6520dcd0033d to finish bootstrapping
I20251024 08:15:01.420675 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e617e2c40edf4e3bac1529df22783efa to finish bootstrapping
I20251024 08:15:01.424461 32207 kudu-admin-test.cc:709] Waiting for Master to see the current replicas...
I20251024 08:15:01.425158 32207 kudu-admin-test.cc:716] Tablet locations:
tablet_locations {
tablet_id: "8ae9089ad39344bc86eb7800e400de64"
DEPRECATED_stale: false
partition {
partition_key_start: ""
partition_key_end: ""
}
interned_replicas {
ts_info_idx: 0
role: FOLLOWER
}
interned_replicas {
ts_info_idx: 1
role: FOLLOWER
}
interned_replicas {
ts_info_idx: 2
role: LEADER
}
}
ts_infos {
permanent_uuid: "e617e2c40edf4e3bac1529df22783efa"
rpc_addresses {
host: "127.31.115.196"
port: 42933
}
}
ts_infos {
permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d"
rpc_addresses {
host: "127.31.115.195"
port: 33929
}
}
ts_infos {
permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9"
rpc_addresses {
host: "127.31.115.193"
port: 43133
}
}
W20251024 08:15:01.430346 32440 tablet.cc:2378] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20251024 08:15:01.456473 32704 tablet.cc:2378] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20251024 08:15:01.594238 367 tablet.cc:2378] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251024 08:15:01.924523 397 consensus_queue.cc:1048] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:01.926998 400 consensus_queue.cc:1048] T 8ae9089ad39344bc86eb7800e400de64 P 1cbadc975b064d5e88f5c479a6ce1ed9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:01.929826 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 32311
I20251024 08:15:01.936595 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 32219
I20251024 08:15:01.942708 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:34799
--webserver_interface=127.31.115.254
--webserver_port=44173
--builtin_ntp_servers=127.31.115.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:34799 with env {}
W20251024 08:15:02.030339 406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:02.030630 406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:02.030668 406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:02.032361 406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:02.032475 406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:02.032501 406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:02.032529 406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:02.034377 406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:34799
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:34799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=44173
--never_fsync=true
--heap_profile_path=/tmp/kudu.406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:02.034754 406 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:02.035027 406 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:02.037648 414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:02.037709 411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:02.037664 412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:02.037904 406 server_base.cc:1047] running on GCE node
I20251024 08:15:02.038152 406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:02.038478 406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:02.039718 406 hybrid_clock.cc:648] HybridClock initialized: now 1761293702039684 us; error 50 us; skew 500 ppm
I20251024 08:15:02.041787 406 webserver.cc:492] Webserver started at http://127.31.115.254:44173/ using document root <none> and password file <none>
I20251024 08:15:02.042075 406 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:02.042167 406 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:02.043958 406 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:02.045130 420 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:02.045454 406 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:02.045583 406 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
uuid: "c62030364a3841969b2af8dc1ed94f73"
format_stamp: "Formatted at 2025-10-24 08:15:00 on dist-test-slave-c0j3"
I20251024 08:15:02.045933 406 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:02.061337 406 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:02.061645 406 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:02.061774 406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:02.065961 406 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:34799
I20251024 08:15:02.066008 472 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:34799 every 8 connection(s)
I20251024 08:15:02.066390 406 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20251024 08:15:02.067703 473 sys_catalog.cc:263] Verifying existing consensus state
I20251024 08:15:02.068173 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 406
I20251024 08:15:02.068295 32207 kudu-admin-test.cc:735] Forcing unsafe config change on tserver e617e2c40edf4e3bac1529df22783efa
I20251024 08:15:02.068411 473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73: Bootstrap starting.
I20251024 08:15:02.069221 32571 heartbeater.cc:499] Master 127.31.115.254:34799 was elected leader, sending a full tablet report...
W20251024 08:15:02.077250 32571 heartbeater.cc:646] Failed to heartbeat to 127.31.115.254:34799 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20251024 08:15:02.078822 473 log.cc:826] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:02.080586 473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:02.080938 473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73: Bootstrap complete.
I20251024 08:15:02.083102 473 raft_consensus.cc:359] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } }
I20251024 08:15:02.083346 473 raft_consensus.cc:740] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62030364a3841969b2af8dc1ed94f73, State: Initialized, Role: FOLLOWER
I20251024 08:15:02.083487 473 consensus_queue.cc:260] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } }
I20251024 08:15:02.083554 473 raft_consensus.cc:399] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:02.083608 473 raft_consensus.cc:493] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:02.083671 473 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:02.084673 473 raft_consensus.cc:515] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } }
I20251024 08:15:02.084786 473 leader_election.cc:304] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c62030364a3841969b2af8dc1ed94f73; no voters:
I20251024 08:15:02.084975 473 leader_election.cc:290] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [CANDIDATE]: Term 2 election: Requested vote from peers
I20251024 08:15:02.085052 479 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 2 FOLLOWER]: Leader election won for term 2
I20251024 08:15:02.085179 479 raft_consensus.cc:697] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [term 2 LEADER]: Becoming Leader. State: Replica: c62030364a3841969b2af8dc1ed94f73, State: Running, Role: LEADER
I20251024 08:15:02.085281 473 sys_catalog.cc:565] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:02.085457 479 consensus_queue.cc:237] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } }
I20251024 08:15:02.085809 479 sys_catalog.cc:455] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c62030364a3841969b2af8dc1ed94f73. Latest consensus state: current_term: 2 leader_uuid: "c62030364a3841969b2af8dc1ed94f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } } }
I20251024 08:15:02.085829 481 sys_catalog.cc:455] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c62030364a3841969b2af8dc1ed94f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c62030364a3841969b2af8dc1ed94f73" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 34799 } } }
I20251024 08:15:02.086027 479 sys_catalog.cc:458] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:02.086041 481 sys_catalog.cc:458] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:02.086285 488 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:02.087031 488 catalog_manager.cc:679] Loaded metadata for table TestTable [id=5fa9fdc0a1314328b3698870c7c60564]
I20251024 08:15:02.087278 488 tablet_loader.cc:96] loaded metadata for tablet 8ae9089ad39344bc86eb7800e400de64 (table TestTable [id=5fa9fdc0a1314328b3698870c7c60564])
I20251024 08:15:02.087353 488 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:02.087433 488 catalog_manager.cc:1269] Loaded cluster ID: 610f182c481144acac15c7e1d71b2551
I20251024 08:15:02.087466 488 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:02.088755 488 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:02.088964 488 catalog_manager.cc:6033] T 00000000000000000000000000000000 P c62030364a3841969b2af8dc1ed94f73: Loaded TSK: 0
I20251024 08:15:02.089207 488 catalog_manager.cc:1524] Initializing in-progress tserver states...
W20251024 08:15:02.157562 475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:02.157778 475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:02.159479 475 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20251024 08:15:02.173449 320 tablet_service.cc:2003] Received UnsafeChangeConfig RPC: dest_uuid: "e617e2c40edf4e3bac1529df22783efa"
tablet_id: "8ae9089ad39344bc86eb7800e400de64"
caller_id: "kudu-tools"
new_config {
peers {
permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d"
}
peers {
permanent_uuid: "e617e2c40edf4e3bac1529df22783efa"
}
}
from {username='slave'} at 127.0.0.1:42158
W20251024 08:15:02.173686 320 raft_consensus.cc:2218] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 1 FOLLOWER]: PROCEEDING WITH UNSAFE CONFIG CHANGE ON THIS SERVER, COMMITTED CONFIG: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }NEW CONFIG: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } unsafe_config_change: true
I20251024 08:15:02.173772 320 raft_consensus.cc:3060] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:02.178705 437 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" instance_seqno: 1761293701341391) as {username='slave'} at 127.31.115.196:43215; Asking this server to re-register.
I20251024 08:15:02.179121 366 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:02.179215 366 heartbeater.cc:507] Master 127.31.115.254:34799 requested a full tablet report, sending...
I20251024 08:15:02.179653 437 ts_manager.cc:194] Registered new tserver with Master: e617e2c40edf4e3bac1529df22783efa (127.31.115.196:42933)
I20251024 08:15:02.930744 437 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" instance_seqno: 1761293701203351) as {username='slave'} at 127.31.115.195:43445; Asking this server to re-register.
I20251024 08:15:02.931218 32703 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:02.931334 32703 heartbeater.cc:507] Master 127.31.115.254:34799 requested a full tablet report, sending...
I20251024 08:15:02.931681 437 ts_manager.cc:194] Registered new tserver with Master: 8fa56def48344ca49f9a6520dcd0033d (127.31.115.195:33929)
I20251024 08:15:03.077479 32571 heartbeater.cc:499] Master 127.31.115.254:34799 was elected leader, sending a full tablet report...
I20251024 08:15:03.078022 437 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" instance_seqno: 1761293701057909) as {username='slave'} at 127.31.115.194:34377; Asking this server to re-register.
I20251024 08:15:03.078400 32571 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:03.078485 32571 heartbeater.cc:507] Master 127.31.115.254:34799 requested a full tablet report, sending...
I20251024 08:15:03.078841 437 ts_manager.cc:194] Registered new tserver with Master: cb057c9691e74d15a4e558a7db6f302f (127.31.115.194:39567)
I20251024 08:15:03.426956 517 raft_consensus.cc:493] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1cbadc975b064d5e88f5c479a6ce1ed9)
I20251024 08:15:03.427142 517 raft_consensus.cc:515] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } }
I20251024 08:15:03.427547 517 leader_election.cc:290] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e617e2c40edf4e3bac1529df22783efa (127.31.115.196:42933), 1cbadc975b064d5e88f5c479a6ce1ed9 (127.31.115.193:43133)
W20251024 08:15:03.428077 32589 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.115.193:43133: connect: Connection refused (error 111)
W20251024 08:15:03.428723 32589 leader_election.cc:336] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1cbadc975b064d5e88f5c479a6ce1ed9 (127.31.115.193:43133): Network error: Client connection negotiation failed: client connection to 127.31.115.193:43133: connect: Connection refused (error 111)
I20251024 08:15:03.431165 320 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ae9089ad39344bc86eb7800e400de64" candidate_uuid: "8fa56def48344ca49f9a6520dcd0033d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e617e2c40edf4e3bac1529df22783efa" is_pre_election: true
I20251024 08:15:03.431595 32591 leader_election.cc:304] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fa56def48344ca49f9a6520dcd0033d; no voters: 1cbadc975b064d5e88f5c479a6ce1ed9, e617e2c40edf4e3bac1529df22783efa
I20251024 08:15:03.431782 517 raft_consensus.cc:2749] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251024 08:15:03.676056 521 raft_consensus.cc:493] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader kudu-tools)
I20251024 08:15:03.676173 521 raft_consensus.cc:515] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } unsafe_config_change: true
I20251024 08:15:03.676494 521 leader_election.cc:290] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fa56def48344ca49f9a6520dcd0033d (127.31.115.195:33929)
I20251024 08:15:03.679654 32657 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ae9089ad39344bc86eb7800e400de64" candidate_uuid: "e617e2c40edf4e3bac1529df22783efa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fa56def48344ca49f9a6520dcd0033d" is_pre_election: true
I20251024 08:15:03.679786 32657 raft_consensus.cc:2468] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e617e2c40edf4e3bac1529df22783efa in term 1.
I20251024 08:15:03.680019 32721 leader_election.cc:304] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 8fa56def48344ca49f9a6520dcd0033d, e617e2c40edf4e3bac1529df22783efa; no voters:
I20251024 08:15:03.680169 521 raft_consensus.cc:2804] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251024 08:15:03.680238 521 raft_consensus.cc:493] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 2 FOLLOWER]: Starting leader election (detected failure of leader kudu-tools)
I20251024 08:15:03.680256 521 raft_consensus.cc:3060] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 2 FOLLOWER]: Advancing to term 3
I20251024 08:15:03.680902 521 raft_consensus.cc:515] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } unsafe_config_change: true
I20251024 08:15:03.681017 521 leader_election.cc:290] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [CANDIDATE]: Term 3 election: Requested vote from peers 8fa56def48344ca49f9a6520dcd0033d (127.31.115.195:33929)
I20251024 08:15:03.681248 32657 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ae9089ad39344bc86eb7800e400de64" candidate_uuid: "e617e2c40edf4e3bac1529df22783efa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fa56def48344ca49f9a6520dcd0033d"
I20251024 08:15:03.681318 32657 raft_consensus.cc:3060] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 1 FOLLOWER]: Advancing to term 3
I20251024 08:15:03.681897 32657 raft_consensus.cc:2468] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e617e2c40edf4e3bac1529df22783efa in term 3.
I20251024 08:15:03.682085 32721 leader_election.cc:304] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 8fa56def48344ca49f9a6520dcd0033d, e617e2c40edf4e3bac1529df22783efa; no voters:
I20251024 08:15:03.682183 521 raft_consensus.cc:2804] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 3 FOLLOWER]: Leader election won for term 3
I20251024 08:15:03.682332 521 raft_consensus.cc:697] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 3 LEADER]: Becoming Leader. State: Replica: e617e2c40edf4e3bac1529df22783efa, State: Running, Role: LEADER
I20251024 08:15:03.682417 521 consensus_queue.cc:237] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } unsafe_config_change: true
I20251024 08:15:03.683228 437 catalog_manager.cc:5649] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa reported cstate change: term changed from 1 to 3, leader changed from 1cbadc975b064d5e88f5c479a6ce1ed9 (127.31.115.193) to e617e2c40edf4e3bac1529df22783efa (127.31.115.196), now has a pending config: VOTER e617e2c40edf4e3bac1529df22783efa (127.31.115.196), VOTER 8fa56def48344ca49f9a6520dcd0033d (127.31.115.195). New cstate: current_term: 3 leader_uuid: "e617e2c40edf4e3bac1529df22783efa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cbadc975b064d5e88f5c479a6ce1ed9" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 43133 } } } pending_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } unsafe_config_change: true }
I20251024 08:15:04.128377 32657 raft_consensus.cc:1275] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 3 FOLLOWER]: Refusing update from remote peer e617e2c40edf4e3bac1529df22783efa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251024 08:15:04.128832 521 consensus_queue.cc:1048] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20251024 08:15:04.130213 524 raft_consensus.cc:2955] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 3 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, VOTER 1cbadc975b064d5e88f5c479a6ce1ed9 (127.31.115.193) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } unsafe_config_change: true }
I20251024 08:15:04.130366 32657 raft_consensus.cc:2955] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 3 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, VOTER 1cbadc975b064d5e88f5c479a6ce1ed9 (127.31.115.193) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } unsafe_config_change: true }
I20251024 08:15:04.131865 437 catalog_manager.cc:5649] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d reported cstate change: config changed from index -1 to 2, VOTER 1cbadc975b064d5e88f5c479a6ce1ed9 (127.31.115.193) evicted, no longer has a pending config: VOTER e617e2c40edf4e3bac1529df22783efa (127.31.115.196), VOTER 8fa56def48344ca49f9a6520dcd0033d (127.31.115.195). New cstate: current_term: 3 leader_uuid: "e617e2c40edf4e3bac1529df22783efa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } unsafe_config_change: true }
W20251024 08:15:04.132855 437 catalog_manager.cc:5841] Failed to send DeleteTablet RPC for tablet 8ae9089ad39344bc86eb7800e400de64 on TS 1cbadc975b064d5e88f5c479a6ce1ed9: Not found: failed to reset TS proxy: Could not find TS for UUID 1cbadc975b064d5e88f5c479a6ce1ed9
I20251024 08:15:04.136987 320 consensus_queue.cc:237] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: true } } unsafe_config_change: true
I20251024 08:15:04.137804 32657 raft_consensus.cc:1275] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 3 FOLLOWER]: Refusing update from remote peer e617e2c40edf4e3bac1529df22783efa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20251024 08:15:04.138098 521 consensus_queue.cc:1048] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251024 08:15:04.139320 524 raft_consensus.cc:2955] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, NON_VOTER cb057c9691e74d15a4e558a7db6f302f (127.31.115.194) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: true } } unsafe_config_change: true }
I20251024 08:15:04.139611 32657 raft_consensus.cc:2955] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, NON_VOTER cb057c9691e74d15a4e558a7db6f302f (127.31.115.194) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: true } } unsafe_config_change: true }
I20251024 08:15:04.140802 423 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 8ae9089ad39344bc86eb7800e400de64 with cas_config_opid_index 2: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251024 08:15:04.140733 436 catalog_manager.cc:5649] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa reported cstate change: config changed from index 2 to 4, NON_VOTER cb057c9691e74d15a4e558a7db6f302f (127.31.115.194) added. New cstate: current_term: 3 leader_uuid: "e617e2c40edf4e3bac1529df22783efa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: true } health_report { overall_health: UNKNOWN } } unsafe_config_change: true }
W20251024 08:15:04.141556 32721 consensus_peers.cc:597] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa -> Peer cb057c9691e74d15a4e558a7db6f302f (127.31.115.194:39567): Couldn't send request to peer cb057c9691e74d15a4e558a7db6f302f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae9089ad39344bc86eb7800e400de64. This is attempt 1: this message will repeat every 5th retry.
W20251024 08:15:04.185295 422 catalog_manager.cc:4767] Async tablet task DeleteTablet RPC for tablet 8ae9089ad39344bc86eb7800e400de64 on TS 1cbadc975b064d5e88f5c479a6ce1ed9 failed: Not found: failed to reset TS proxy: Could not find TS for UUID 1cbadc975b064d5e88f5c479a6ce1ed9
I20251024 08:15:04.543613 539 ts_tablet_manager.cc:933] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: Initiating tablet copy from peer e617e2c40edf4e3bac1529df22783efa (127.31.115.196:42933)
I20251024 08:15:04.543889 539 tablet_copy_client.cc:323] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: tablet copy: Beginning tablet copy session from remote peer at address 127.31.115.196:42933
I20251024 08:15:04.547019 340 tablet_copy_service.cc:140] P e617e2c40edf4e3bac1529df22783efa: Received BeginTabletCopySession request for tablet 8ae9089ad39344bc86eb7800e400de64 from peer cb057c9691e74d15a4e558a7db6f302f ({username='slave'} at 127.31.115.194:55861)
I20251024 08:15:04.547115 340 tablet_copy_service.cc:161] P e617e2c40edf4e3bac1529df22783efa: Beginning new tablet copy session on tablet 8ae9089ad39344bc86eb7800e400de64 from peer cb057c9691e74d15a4e558a7db6f302f at {username='slave'} at 127.31.115.194:55861: session id = cb057c9691e74d15a4e558a7db6f302f-8ae9089ad39344bc86eb7800e400de64
I20251024 08:15:04.547662 340 tablet_copy_source_session.cc:215] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa: Tablet Copy: opened 0 blocks and 1 log segments
I20251024 08:15:04.548552 539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae9089ad39344bc86eb7800e400de64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:04.550920 539 tablet_copy_client.cc:806] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: tablet copy: Starting download of 0 data blocks...
I20251024 08:15:04.551064 539 tablet_copy_client.cc:670] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: tablet copy: Starting download of 1 WAL segments...
I20251024 08:15:04.552107 539 tablet_copy_client.cc:538] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251024 08:15:04.552840 539 tablet_bootstrap.cc:492] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: Bootstrap starting.
I20251024 08:15:04.554054 539 log.cc:826] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:04.555119 539 tablet_bootstrap.cc:492] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:04.555334 539 tablet_bootstrap.cc:492] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: Bootstrap complete.
I20251024 08:15:04.555470 539 ts_tablet_manager.cc:1403] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251024 08:15:04.557071 539 raft_consensus.cc:359] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: true } } unsafe_config_change: true
I20251024 08:15:04.557231 539 raft_consensus.cc:740] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: cb057c9691e74d15a4e558a7db6f302f, State: Initialized, Role: LEARNER
I20251024 08:15:04.557283 539 consensus_queue.cc:260] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: true } } unsafe_config_change: true
I20251024 08:15:04.557541 539 ts_tablet_manager.cc:1434] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20251024 08:15:04.557921 340 tablet_copy_service.cc:342] P e617e2c40edf4e3bac1529df22783efa: Request end of tablet copy session cb057c9691e74d15a4e558a7db6f302f-8ae9089ad39344bc86eb7800e400de64 received from {username='slave'} at 127.31.115.194:55861
I20251024 08:15:04.557992 340 tablet_copy_service.cc:434] P e617e2c40edf4e3bac1529df22783efa: ending tablet copy session cb057c9691e74d15a4e558a7db6f302f-8ae9089ad39344bc86eb7800e400de64 on tablet 8ae9089ad39344bc86eb7800e400de64 with peer cb057c9691e74d15a4e558a7db6f302f
I20251024 08:15:04.993585 32525 raft_consensus.cc:1217] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4] Dedup: 3.4->[]
W20251024 08:15:05.350468 422 catalog_manager.cc:4767] Async tablet task DeleteTablet RPC for tablet 8ae9089ad39344bc86eb7800e400de64 on TS 1cbadc975b064d5e88f5c479a6ce1ed9 failed: Not found: failed to reset TS proxy: Could not find TS for UUID 1cbadc975b064d5e88f5c479a6ce1ed9
I20251024 08:15:05.481595 538 raft_consensus.cc:1064] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa: attempting to promote NON_VOTER cb057c9691e74d15a4e558a7db6f302f to VOTER
I20251024 08:15:05.481859 538 consensus_queue.cc:237] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: false } } unsafe_config_change: true
I20251024 08:15:05.482445 32657 raft_consensus.cc:1275] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 3 FOLLOWER]: Refusing update from remote peer e617e2c40edf4e3bac1529df22783efa: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20251024 08:15:05.482478 32525 raft_consensus.cc:1275] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f [term 3 LEARNER]: Refusing update from remote peer e617e2c40edf4e3bac1529df22783efa: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20251024 08:15:05.482715 538 consensus_queue.cc:1048] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251024 08:15:05.482846 544 consensus_queue.cc:1048] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251024 08:15:05.483639 546 raft_consensus.cc:2955] T 8ae9089ad39344bc86eb7800e400de64 P e617e2c40edf4e3bac1529df22783efa [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cb057c9691e74d15a4e558a7db6f302f (127.31.115.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: false } } unsafe_config_change: true }
I20251024 08:15:05.483778 32657 raft_consensus.cc:2955] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cb057c9691e74d15a4e558a7db6f302f (127.31.115.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: false } } unsafe_config_change: true }
I20251024 08:15:05.485176 32525 raft_consensus.cc:2955] T 8ae9089ad39344bc86eb7800e400de64 P cb057c9691e74d15a4e558a7db6f302f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cb057c9691e74d15a4e558a7db6f302f (127.31.115.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: false } } unsafe_config_change: true }
I20251024 08:15:05.485394 436 catalog_manager.cc:5649] T 8ae9089ad39344bc86eb7800e400de64 P 8fa56def48344ca49f9a6520dcd0033d reported cstate change: config changed from index 4 to 5, cb057c9691e74d15a4e558a7db6f302f (127.31.115.194) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "e617e2c40edf4e3bac1529df22783efa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e617e2c40edf4e3bac1529df22783efa" member_type: VOTER last_known_addr { host: "127.31.115.196" port: 42933 } } peers { permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 33929 } } peers { permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 39567 } attrs { promote: false } } unsafe_config_change: true }
I20251024 08:15:05.595582 32207 kudu-admin-test.cc:751] Waiting for Master to see new config...
I20251024 08:15:05.599926 32207 kudu-admin-test.cc:756] Tablet locations:
tablet_locations {
tablet_id: "8ae9089ad39344bc86eb7800e400de64"
DEPRECATED_stale: false
partition {
partition_key_start: ""
partition_key_end: ""
}
interned_replicas {
ts_info_idx: 0
role: LEADER
}
interned_replicas {
ts_info_idx: 1
role: FOLLOWER
}
interned_replicas {
ts_info_idx: 2
role: FOLLOWER
}
}
ts_infos {
permanent_uuid: "e617e2c40edf4e3bac1529df22783efa"
rpc_addresses {
host: "127.31.115.196"
port: 42933
}
}
ts_infos {
permanent_uuid: "8fa56def48344ca49f9a6520dcd0033d"
rpc_addresses {
host: "127.31.115.195"
port: 33929
}
}
ts_infos {
permanent_uuid: "cb057c9691e74d15a4e558a7db6f302f"
rpc_addresses {
host: "127.31.115.194"
port: 39567
}
}
I20251024 08:15:05.600395 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 32442
I20251024 08:15:05.606786 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 32574
I20251024 08:15:05.613091 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 32706
I20251024 08:15:05.618665 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 406
2025-10-24T08:15:05Z chronyd exiting
[ OK ] AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes (5028 ms)
[ RUN ] AdminCliTest.TestGracefulSpecificLeaderStepDown
I20251024 08:15:05.640751 32207 test_util.cc:276] Using random seed: 631995177
I20251024 08:15:05.641345 32207 ts_itest-base.cc:115] Starting cluster with:
I20251024 08:15:05.641386 32207 ts_itest-base.cc:116] --------------
I20251024 08:15:05.641397 32207 ts_itest-base.cc:117] 3 tablet servers
I20251024 08:15:05.641408 32207 ts_itest-base.cc:118] 3 replicas per TS
I20251024 08:15:05.641422 32207 ts_itest-base.cc:119] --------------
2025-10-24T08:15:05Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-24T08:15:05Z Disabled control of system clock
I20251024 08:15:05.649544 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:45105
--webserver_interface=127.31.115.254
--webserver_port=0
--builtin_ntp_servers=127.31.115.212:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:45105
--catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {}
W20251024 08:15:05.734935 563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:05.735172 563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:05.735227 563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:05.736891 563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:05.736981 563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:05.737010 563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:05.737035 563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:05.738945 563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:45105
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:45105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:05.739317 563 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:05.739599 563 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:05.742591 571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:05.742656 568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:05.742671 569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:05.742983 563 server_base.cc:1047] running on GCE node
I20251024 08:15:05.743296 563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:05.743688 563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:05.744918 563 hybrid_clock.cc:648] HybridClock initialized: now 1761293705744888 us; error 53 us; skew 500 ppm
I20251024 08:15:05.746591 563 webserver.cc:492] Webserver started at http://127.31.115.254:35647/ using document root <none> and password file <none>
I20251024 08:15:05.746865 563 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:05.746928 563 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:05.747085 563 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:05.748129 563 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "8e5173f2629947db9170b5c4d322baf3"
format_stamp: "Formatted at 2025-10-24 08:15:05 on dist-test-slave-c0j3"
I20251024 08:15:05.748533 563 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "8e5173f2629947db9170b5c4d322baf3"
format_stamp: "Formatted at 2025-10-24 08:15:05 on dist-test-slave-c0j3"
I20251024 08:15:05.750061 563 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20251024 08:15:05.751119 577 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:05.751327 563 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:05.751438 563 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
uuid: "8e5173f2629947db9170b5c4d322baf3"
format_stamp: "Formatted at 2025-10-24 08:15:05 on dist-test-slave-c0j3"
I20251024 08:15:05.751528 563 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:05.760818 563 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:05.761174 563 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:05.761317 563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:05.766053 563 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:45105
I20251024 08:15:05.766119 629 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:45105 every 8 connection(s)
I20251024 08:15:05.766582 563 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20251024 08:15:05.767269 630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:05.769877 630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3: Bootstrap starting.
I20251024 08:15:05.770629 630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:05.770931 630 log.cc:826] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:05.771734 630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3: No bootstrap required, opened a new log
I20251024 08:15:05.773097 630 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5173f2629947db9170b5c4d322baf3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45105 } }
I20251024 08:15:05.773274 630 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:05.773308 630 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e5173f2629947db9170b5c4d322baf3, State: Initialized, Role: FOLLOWER
I20251024 08:15:05.773450 630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5173f2629947db9170b5c4d322baf3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45105 } }
I20251024 08:15:05.773516 630 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:05.773559 630 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:05.773617 630 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:05.774248 630 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5173f2629947db9170b5c4d322baf3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45105 } }
I20251024 08:15:05.774376 630 leader_election.cc:304] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e5173f2629947db9170b5c4d322baf3; no voters:
I20251024 08:15:05.774631 630 leader_election.cc:290] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:05.774734 633 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:05.774921 633 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [term 1 LEADER]: Becoming Leader. State: Replica: 8e5173f2629947db9170b5c4d322baf3, State: Running, Role: LEADER
I20251024 08:15:05.774981 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 563
I20251024 08:15:05.775072 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance
I20251024 08:15:05.775039 633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5173f2629947db9170b5c4d322baf3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45105 } }
I20251024 08:15:05.775435 633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e5173f2629947db9170b5c4d322baf3. Latest consensus state: current_term: 1 leader_uuid: "8e5173f2629947db9170b5c4d322baf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5173f2629947db9170b5c4d322baf3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45105 } } }
I20251024 08:15:05.775539 633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:05.775543 630 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:05.775771 637 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:05.776088 634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e5173f2629947db9170b5c4d322baf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5173f2629947db9170b5c4d322baf3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45105 } } }
I20251024 08:15:05.776176 634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:05.776364 637 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:05.778882 637 catalog_manager.cc:1357] Generated new cluster ID: 52aa5e27cea3482a93b5fb4eafbb2020
I20251024 08:15:05.778987 637 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:05.803399 637 catalog_manager.cc:1380] Generated new certificate authority record
I20251024 08:15:05.804112 637 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:05.816073 637 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 8e5173f2629947db9170b5c4d322baf3: Generated new TSK 0
I20251024 08:15:05.816325 637 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251024 08:15:05.822471 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.193:0
--local_ip_for_outbound_sockets=127.31.115.193
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45105
--builtin_ntp_servers=127.31.115.212:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20251024 08:15:05.910492 654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20251024 08:15:05.910773 654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:05.910789 654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:05.910811 654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:05.912530 654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:05.912624 654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.193
I20251024 08:15:05.914476 654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45105
--never_fsync=true
--heap_profile_path=/tmp/kudu.654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.193
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:05.914826 654 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:05.915118 654 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:05.918056 659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:05.918097 660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:05.918107 662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:05.918557 654 server_base.cc:1047] running on GCE node
I20251024 08:15:05.918797 654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:05.919059 654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:05.920225 654 hybrid_clock.cc:648] HybridClock initialized: now 1761293705920197 us; error 57 us; skew 500 ppm
I20251024 08:15:05.921787 654 webserver.cc:492] Webserver started at http://127.31.115.193:46567/ using document root <none> and password file <none>
I20251024 08:15:05.922041 654 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:05.922097 654 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:05.922223 654 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:05.923244 654 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb"
format_stamp: "Formatted at 2025-10-24 08:15:05 on dist-test-slave-c0j3"
I20251024 08:15:05.923599 654 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb"
format_stamp: "Formatted at 2025-10-24 08:15:05 on dist-test-slave-c0j3"
I20251024 08:15:05.925079 654 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20251024 08:15:05.926071 668 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:05.926350 654 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:05.926460 654 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb"
format_stamp: "Formatted at 2025-10-24 08:15:05 on dist-test-slave-c0j3"
I20251024 08:15:05.926587 654 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:05.949601 654 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:05.949898 654 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:05.950021 654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:05.950253 654 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:05.950699 654 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:05.950742 654 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:05.950778 654 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:05.950800 654 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:05.957845 654 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.193:34035
I20251024 08:15:05.957890 781 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.193:34035 every 8 connection(s)
I20251024 08:15:05.958326 654 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20251024 08:15:05.963622 782 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45105
I20251024 08:15:05.963752 782 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:05.963982 782 heartbeater.cc:507] Master 127.31.115.254:45105 requested a full tablet report, sending...
I20251024 08:15:05.964516 594 ts_manager.cc:194] Registered new tserver with Master: d4b32b3e334d4a8c969b29cd4dfe80eb (127.31.115.193:34035)
I20251024 08:15:05.965180 594 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.193:59933
I20251024 08:15:05.968093 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 654
I20251024 08:15:05.968180 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20251024 08:15:05.969559 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.194:0
--local_ip_for_outbound_sockets=127.31.115.194
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45105
--builtin_ntp_servers=127.31.115.212:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20251024 08:15:06.054250 785 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20251024 08:15:06.054483 785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:06.054541 785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:06.054580 785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:06.056272 785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:06.056353 785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.194
I20251024 08:15:06.058174 785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45105
--never_fsync=true
--heap_profile_path=/tmp/kudu.785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.194
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:06.058466 785 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:06.058771 785 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:06.061513 790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:06.061532 791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:06.061517 793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:06.061895 785 server_base.cc:1047] running on GCE node
I20251024 08:15:06.062077 785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:06.062325 785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:06.063481 785 hybrid_clock.cc:648] HybridClock initialized: now 1761293706063465 us; error 32 us; skew 500 ppm
I20251024 08:15:06.065243 785 webserver.cc:492] Webserver started at http://127.31.115.194:41377/ using document root <none> and password file <none>
I20251024 08:15:06.065557 785 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:06.065621 785 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:06.065781 785 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:06.066936 785 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "1af8c483bf9f4e2f9ca549058e2e56a0"
format_stamp: "Formatted at 2025-10-24 08:15:06 on dist-test-slave-c0j3"
I20251024 08:15:06.067309 785 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "1af8c483bf9f4e2f9ca549058e2e56a0"
format_stamp: "Formatted at 2025-10-24 08:15:06 on dist-test-slave-c0j3"
I20251024 08:15:06.068828 785 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20251024 08:15:06.069878 799 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:06.070152 785 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:06.070266 785 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "1af8c483bf9f4e2f9ca549058e2e56a0"
format_stamp: "Formatted at 2025-10-24 08:15:06 on dist-test-slave-c0j3"
I20251024 08:15:06.070356 785 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:06.082593 785 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:06.082863 785 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:06.082957 785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:06.083145 785 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:06.083505 785 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:06.083541 785 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:06.083581 785 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:06.083606 785 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:06.089504 785 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.194:41701
I20251024 08:15:06.089600 913 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.194:41701 every 8 connection(s)
I20251024 08:15:06.089885 785 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20251024 08:15:06.094667 914 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45105
I20251024 08:15:06.094790 914 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:06.094987 914 heartbeater.cc:507] Master 127.31.115.254:45105 requested a full tablet report, sending...
I20251024 08:15:06.095290 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 785
I20251024 08:15:06.095379 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20251024 08:15:06.095445 594 ts_manager.cc:194] Registered new tserver with Master: 1af8c483bf9f4e2f9ca549058e2e56a0 (127.31.115.194:41701)
I20251024 08:15:06.095914 594 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.194:44699
I20251024 08:15:06.096704 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.195:0
--local_ip_for_outbound_sockets=127.31.115.195
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45105
--builtin_ntp_servers=127.31.115.212:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20251024 08:15:06.181120 917 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20251024 08:15:06.181342 917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:06.181372 917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:06.181401 917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:06.183066 917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:06.183158 917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.195
I20251024 08:15:06.184974 917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45105
--never_fsync=true
--heap_profile_path=/tmp/kudu.917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.195
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:06.185289 917 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:06.185566 917 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:06.188681 923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:06.188684 922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:06.188772 925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:06.189033 917 server_base.cc:1047] running on GCE node
I20251024 08:15:06.189294 917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:06.189601 917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:06.190780 917 hybrid_clock.cc:648] HybridClock initialized: now 1761293706190740 us; error 63 us; skew 500 ppm
I20251024 08:15:06.192375 917 webserver.cc:492] Webserver started at http://127.31.115.195:40775/ using document root <none> and password file <none>
I20251024 08:15:06.192628 917 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:06.192682 917 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:06.192809 917 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:06.193743 917 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "9044a34926134b83af1009e825eda4d0"
format_stamp: "Formatted at 2025-10-24 08:15:06 on dist-test-slave-c0j3"
I20251024 08:15:06.194059 917 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "9044a34926134b83af1009e825eda4d0"
format_stamp: "Formatted at 2025-10-24 08:15:06 on dist-test-slave-c0j3"
I20251024 08:15:06.195714 917 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20251024 08:15:06.196607 931 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:06.196849 917 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:06.196928 917 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "9044a34926134b83af1009e825eda4d0"
format_stamp: "Formatted at 2025-10-24 08:15:06 on dist-test-slave-c0j3"
I20251024 08:15:06.197007 917 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:06.216583 917 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:06.216907 917 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:06.217036 917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:06.217263 917 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:06.217618 917 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:06.217653 917 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:06.217691 917 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:06.217712 917 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:06.225333 917 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.195:36897
I20251024 08:15:06.225745 1045 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.195:36897 every 8 connection(s)
I20251024 08:15:06.226115 917 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20251024 08:15:06.231642 1046 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45105
I20251024 08:15:06.231760 1046 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:06.231956 1046 heartbeater.cc:507] Master 127.31.115.254:45105 requested a full tablet report, sending...
I20251024 08:15:06.232043 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 917
I20251024 08:15:06.232134 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20251024 08:15:06.232398 593 ts_manager.cc:194] Registered new tserver with Master: 9044a34926134b83af1009e825eda4d0 (127.31.115.195:36897)
I20251024 08:15:06.232931 593 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.195:45455
I20251024 08:15:06.233188 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:06.239600 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37476:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20251024 08:15:06.240984 593 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251024 08:15:06.247952 716 tablet_service.cc:1505] Processing CreateTablet for tablet d845e1a3134d4b40a8073d7b5a28d6f5 (DEFAULT_TABLE table=TestTable [id=b6ea44927d394fdaa6f260f4ff77ceee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:06.248205 979 tablet_service.cc:1505] Processing CreateTablet for tablet d845e1a3134d4b40a8073d7b5a28d6f5 (DEFAULT_TABLE table=TestTable [id=b6ea44927d394fdaa6f260f4ff77ceee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:06.248378 716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d845e1a3134d4b40a8073d7b5a28d6f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:06.248474 979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d845e1a3134d4b40a8073d7b5a28d6f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:06.248543 848 tablet_service.cc:1505] Processing CreateTablet for tablet d845e1a3134d4b40a8073d7b5a28d6f5 (DEFAULT_TABLE table=TestTable [id=b6ea44927d394fdaa6f260f4ff77ceee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:06.248850 848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d845e1a3134d4b40a8073d7b5a28d6f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:06.251294 1060 tablet_bootstrap.cc:492] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0: Bootstrap starting.
I20251024 08:15:06.251722 1062 tablet_bootstrap.cc:492] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb: Bootstrap starting.
I20251024 08:15:06.252260 1060 tablet_bootstrap.cc:654] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:06.252291 1062 tablet_bootstrap.cc:654] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:06.252539 1062 log.cc:826] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:06.252552 1060 log.cc:826] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:06.252861 1061 tablet_bootstrap.cc:492] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0: Bootstrap starting.
I20251024 08:15:06.253290 1062 tablet_bootstrap.cc:492] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb: No bootstrap required, opened a new log
I20251024 08:15:06.253334 1060 tablet_bootstrap.cc:492] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0: No bootstrap required, opened a new log
I20251024 08:15:06.253374 1062 ts_tablet_manager.cc:1403] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:06.253386 1060 ts_tablet_manager.cc:1403] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:06.253439 1061 tablet_bootstrap.cc:654] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:06.253691 1061 log.cc:826] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:06.254616 1061 tablet_bootstrap.cc:492] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0: No bootstrap required, opened a new log
I20251024 08:15:06.254686 1061 ts_tablet_manager.cc:1403] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:06.255149 1062 raft_consensus.cc:359] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:06.255324 1062 raft_consensus.cc:740] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b32b3e334d4a8c969b29cd4dfe80eb, State: Initialized, Role: FOLLOWER
I20251024 08:15:06.255393 1062 consensus_queue.cc:260] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:06.255399 1060 raft_consensus.cc:359] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:06.255532 1060 raft_consensus.cc:740] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1af8c483bf9f4e2f9ca549058e2e56a0, State: Initialized, Role: FOLLOWER
I20251024 08:15:06.255590 1060 consensus_queue.cc:260] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:06.255770 1062 ts_tablet_manager.cc:1434] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251024 08:15:06.255849 1060 ts_tablet_manager.cc:1434] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251024 08:15:06.255903 782 heartbeater.cc:499] Master 127.31.115.254:45105 was elected leader, sending a full tablet report...
I20251024 08:15:06.255949 914 heartbeater.cc:499] Master 127.31.115.254:45105 was elected leader, sending a full tablet report...
I20251024 08:15:06.256896 1061 raft_consensus.cc:359] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:06.257341 1061 raft_consensus.cc:740] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9044a34926134b83af1009e825eda4d0, State: Initialized, Role: FOLLOWER
I20251024 08:15:06.257428 1061 consensus_queue.cc:260] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:06.257779 1061 ts_tablet_manager.cc:1434] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20251024 08:15:06.257838 1046 heartbeater.cc:499] Master 127.31.115.254:45105 was elected leader, sending a full tablet report...
I20251024 08:15:06.262732 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:06.263285 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver d4b32b3e334d4a8c969b29cd4dfe80eb to finish bootstrapping
I20251024 08:15:06.267050 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 1af8c483bf9f4e2f9ca549058e2e56a0 to finish bootstrapping
I20251024 08:15:06.270629 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 9044a34926134b83af1009e825eda4d0 to finish bootstrapping
I20251024 08:15:06.284693 868 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "d845e1a3134d4b40a8073d7b5a28d6f5"
dest_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0"
from {username='slave'} at 127.0.0.1:34388
I20251024 08:15:06.284839 868 raft_consensus.cc:493] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20251024 08:15:06.284894 868 raft_consensus.cc:3060] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:06.285921 868 raft_consensus.cc:515] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:06.286273 868 leader_election.cc:290] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [CANDIDATE]: Term 1 election: Requested vote from peers 9044a34926134b83af1009e825eda4d0 (127.31.115.195:36897), d4b32b3e334d4a8c969b29cd4dfe80eb (127.31.115.193:34035)
I20251024 08:15:06.287945 32207 cluster_itest_util.cc:257] Not converged past 1 yet: 0.0 0.0 0.0
I20251024 08:15:06.289435 736 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d845e1a3134d4b40a8073d7b5a28d6f5" candidate_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb"
I20251024 08:15:06.289525 1000 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d845e1a3134d4b40a8073d7b5a28d6f5" candidate_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9044a34926134b83af1009e825eda4d0"
I20251024 08:15:06.289618 736 raft_consensus.cc:3060] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:06.289630 1000 raft_consensus.cc:3060] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:06.291026 1000 raft_consensus.cc:2468] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1af8c483bf9f4e2f9ca549058e2e56a0 in term 1.
I20251024 08:15:06.291026 736 raft_consensus.cc:2468] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1af8c483bf9f4e2f9ca549058e2e56a0 in term 1.
I20251024 08:15:06.291360 801 leader_election.cc:304] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1af8c483bf9f4e2f9ca549058e2e56a0, d4b32b3e334d4a8c969b29cd4dfe80eb; no voters:
I20251024 08:15:06.291548 1067 raft_consensus.cc:2804] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:06.291754 1067 raft_consensus.cc:697] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 1 LEADER]: Becoming Leader. State: Replica: 1af8c483bf9f4e2f9ca549058e2e56a0, State: Running, Role: LEADER
I20251024 08:15:06.291877 1067 consensus_queue.cc:237] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:06.292814 594 catalog_manager.cc:5649] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1af8c483bf9f4e2f9ca549058e2e56a0 (127.31.115.194). New cstate: current_term: 1 leader_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } health_report { overall_health: UNKNOWN } } }
W20251024 08:15:06.340605 915 tablet.cc:2378] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251024 08:15:06.389979 32207 cluster_itest_util.cc:257] Not converged past 1 yet: 1.1 0.0 0.0
W20251024 08:15:06.459132 783 tablet.cc:2378] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20251024 08:15:06.476806 1047 tablet.cc:2378] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251024 08:15:06.592092 32207 cluster_itest_util.cc:257] Not converged past 1 yet: 1.1 0.0 0.0
I20251024 08:15:06.693811 1067 consensus_queue.cc:1048] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:06.696812 1067 consensus_queue.cc:1048] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:07.004944 868 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "d845e1a3134d4b40a8073d7b5a28d6f5"
dest_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:34398
I20251024 08:15:07.005065 868 raft_consensus.cc:606] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 1 LEADER]: Received request to transfer leadership
I20251024 08:15:07.119835 1071 raft_consensus.cc:993] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0: : Instructing follower d4b32b3e334d4a8c969b29cd4dfe80eb to start an election
I20251024 08:15:07.119954 1071 raft_consensus.cc:1081] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 1 LEADER]: Signalling peer d4b32b3e334d4a8c969b29cd4dfe80eb to start an election
I20251024 08:15:07.120189 736 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "d845e1a3134d4b40a8073d7b5a28d6f5"
dest_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb"
from {username='slave'} at 127.31.115.194:41187
I20251024 08:15:07.120313 736 raft_consensus.cc:493] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251024 08:15:07.120337 736 raft_consensus.cc:3060] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:07.121009 736 raft_consensus.cc:515] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:07.121309 736 leader_election.cc:290] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [CANDIDATE]: Term 2 election: Requested vote from peers 1af8c483bf9f4e2f9ca549058e2e56a0 (127.31.115.194:41701), 9044a34926134b83af1009e825eda4d0 (127.31.115.195:36897)
I20251024 08:15:07.125121 868 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d845e1a3134d4b40a8073d7b5a28d6f5" candidate_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0"
I20251024 08:15:07.125121 1000 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d845e1a3134d4b40a8073d7b5a28d6f5" candidate_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9044a34926134b83af1009e825eda4d0"
I20251024 08:15:07.125320 1000 raft_consensus.cc:3060] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0 [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:07.125348 868 raft_consensus.cc:3055] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 1 LEADER]: Stepping down as leader of term 1
I20251024 08:15:07.125378 868 raft_consensus.cc:740] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1af8c483bf9f4e2f9ca549058e2e56a0, State: Running, Role: LEADER
I20251024 08:15:07.125433 868 consensus_queue.cc:260] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:07.125589 868 raft_consensus.cc:3060] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:07.126405 1000 raft_consensus.cc:2468] T d845e1a3134d4b40a8073d7b5a28d6f5 P 9044a34926134b83af1009e825eda4d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4b32b3e334d4a8c969b29cd4dfe80eb in term 2.
I20251024 08:15:07.126451 868 raft_consensus.cc:2468] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4b32b3e334d4a8c969b29cd4dfe80eb in term 2.
I20251024 08:15:07.126797 670 leader_election.cc:304] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1af8c483bf9f4e2f9ca549058e2e56a0, 9044a34926134b83af1009e825eda4d0, d4b32b3e334d4a8c969b29cd4dfe80eb; no voters:
I20251024 08:15:07.127012 1066 raft_consensus.cc:2804] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [term 2 FOLLOWER]: Leader election won for term 2
I20251024 08:15:07.127168 1066 raft_consensus.cc:697] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [term 2 LEADER]: Becoming Leader. State: Replica: d4b32b3e334d4a8c969b29cd4dfe80eb, State: Running, Role: LEADER
I20251024 08:15:07.127272 1066 consensus_queue.cc:237] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } }
I20251024 08:15:07.128162 594 catalog_manager.cc:5649] T d845e1a3134d4b40a8073d7b5a28d6f5 P d4b32b3e334d4a8c969b29cd4dfe80eb reported cstate change: term changed from 1 to 2, leader changed from 1af8c483bf9f4e2f9ca549058e2e56a0 (127.31.115.194) to d4b32b3e334d4a8c969b29cd4dfe80eb (127.31.115.193). New cstate: current_term: 2 leader_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9044a34926134b83af1009e825eda4d0" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4b32b3e334d4a8c969b29cd4dfe80eb" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34035 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:07.240047 868 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "d845e1a3134d4b40a8073d7b5a28d6f5"
dest_uuid: "1af8c483bf9f4e2f9ca549058e2e56a0"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:34402
I20251024 08:15:07.240159 868 raft_consensus.cc:606] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 2 FOLLOWER]: Received request to transfer leadership
I20251024 08:15:07.240180 868 raft_consensus.cc:614] T d845e1a3134d4b40a8073d7b5a28d6f5 P 1af8c483bf9f4e2f9ca549058e2e56a0 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20251024 08:15:07.297566 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 654
I20251024 08:15:07.303813 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 785
I20251024 08:15:07.309018 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 917
I20251024 08:15:07.314419 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 563
2025-10-24T08:15:07Z chronyd exiting
[ OK ] AdminCliTest.TestGracefulSpecificLeaderStepDown (1694 ms)
[ RUN ] AdminCliTest.TestDescribeTableColumnFlags
I20251024 08:15:07.335337 32207 test_util.cc:276] Using random seed: 633689762
I20251024 08:15:07.335947 32207 ts_itest-base.cc:115] Starting cluster with:
I20251024 08:15:07.335989 32207 ts_itest-base.cc:116] --------------
I20251024 08:15:07.336009 32207 ts_itest-base.cc:117] 3 tablet servers
I20251024 08:15:07.336027 32207 ts_itest-base.cc:118] 3 replicas per TS
I20251024 08:15:07.336054 32207 ts_itest-base.cc:119] --------------
2025-10-24T08:15:07Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-24T08:15:07Z Disabled control of system clock
I20251024 08:15:07.344565 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:45705
--webserver_interface=127.31.115.254
--webserver_port=0
--builtin_ntp_servers=127.31.115.212:45209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:45705 with env {}
W20251024 08:15:07.430269 1125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:07.430485 1125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:07.430536 1125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:07.432206 1125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:07.432286 1125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:07.432301 1125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:07.432314 1125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:07.434185 1125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:45209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:45705
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:45705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.1125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:07.434542 1125 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:07.434809 1125 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:07.437638 1131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:07.437590 1133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:07.437742 1125 server_base.cc:1047] running on GCE node
W20251024 08:15:07.437588 1130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:07.438117 1125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:07.438408 1125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:07.439568 1125 hybrid_clock.cc:648] HybridClock initialized: now 1761293707439562 us; error 70 us; skew 500 ppm
I20251024 08:15:07.441047 1125 webserver.cc:492] Webserver started at http://127.31.115.254:34661/ using document root <none> and password file <none>
I20251024 08:15:07.441299 1125 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:07.441351 1125 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:07.441504 1125 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:07.442574 1125 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "fdf8feaa51bc4a27adc71b741f8faeb0"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.442957 1125 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "fdf8feaa51bc4a27adc71b741f8faeb0"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.444365 1125 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20251024 08:15:07.445281 1139 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:07.445498 1125 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:07.445585 1125 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
uuid: "fdf8feaa51bc4a27adc71b741f8faeb0"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.445665 1125 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:07.463068 1125 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:07.463434 1125 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:07.463577 1125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:07.467865 1125 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:45705
I20251024 08:15:07.467973 1192 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:45705 every 8 connection(s)
I20251024 08:15:07.468353 1125 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20251024 08:15:07.469237 1193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:07.470083 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 1125
I20251024 08:15:07.470191 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance
I20251024 08:15:07.473462 1193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0: Bootstrap starting.
I20251024 08:15:07.474421 1193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:07.474846 1193 log.cc:826] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:07.476054 1193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0: No bootstrap required, opened a new log
I20251024 08:15:07.477653 1193 raft_consensus.cc:359] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf8feaa51bc4a27adc71b741f8faeb0" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45705 } }
I20251024 08:15:07.477816 1193 raft_consensus.cc:385] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:07.477839 1193 raft_consensus.cc:740] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdf8feaa51bc4a27adc71b741f8faeb0, State: Initialized, Role: FOLLOWER
I20251024 08:15:07.477936 1193 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf8feaa51bc4a27adc71b741f8faeb0" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45705 } }
I20251024 08:15:07.478003 1193 raft_consensus.cc:399] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:07.478039 1193 raft_consensus.cc:493] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:07.478092 1193 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:07.478796 1193 raft_consensus.cc:515] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf8feaa51bc4a27adc71b741f8faeb0" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45705 } }
I20251024 08:15:07.478915 1193 leader_election.cc:304] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdf8feaa51bc4a27adc71b741f8faeb0; no voters:
I20251024 08:15:07.479095 1193 leader_election.cc:290] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:07.479188 1198 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:07.479346 1193 sys_catalog.cc:565] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:07.479360 1198 raft_consensus.cc:697] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [term 1 LEADER]: Becoming Leader. State: Replica: fdf8feaa51bc4a27adc71b741f8faeb0, State: Running, Role: LEADER
I20251024 08:15:07.479508 1198 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf8feaa51bc4a27adc71b741f8faeb0" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45705 } }
I20251024 08:15:07.479872 1200 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdf8feaa51bc4a27adc71b741f8faeb0. Latest consensus state: current_term: 1 leader_uuid: "fdf8feaa51bc4a27adc71b741f8faeb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf8feaa51bc4a27adc71b741f8faeb0" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45705 } } }
I20251024 08:15:07.479985 1200 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:07.479892 1199 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdf8feaa51bc4a27adc71b741f8faeb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf8feaa51bc4a27adc71b741f8faeb0" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45705 } } }
I20251024 08:15:07.480085 1199 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:07.480224 1206 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:07.480695 1206 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:07.482371 1206 catalog_manager.cc:1357] Generated new cluster ID: 0e25c50dee5f4727870fa9d057d08859
I20251024 08:15:07.482442 1206 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:07.501156 1206 catalog_manager.cc:1380] Generated new certificate authority record
I20251024 08:15:07.502023 1206 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:07.509156 1206 catalog_manager.cc:6022] T 00000000000000000000000000000000 P fdf8feaa51bc4a27adc71b741f8faeb0: Generated new TSK 0
I20251024 08:15:07.509400 1206 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251024 08:15:07.516968 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.193:0
--local_ip_for_outbound_sockets=127.31.115.193
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45705
--builtin_ntp_servers=127.31.115.212:45209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20251024 08:15:07.607486 1217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:07.607712 1217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:07.607753 1217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:07.609440 1217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:07.609532 1217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.193
I20251024 08:15:07.611399 1217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:45209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45705
--never_fsync=true
--heap_profile_path=/tmp/kudu.1217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.193
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:07.611706 1217 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:07.611992 1217 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:07.615186 1225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:07.615231 1222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:07.615270 1223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:07.616739 1217 server_base.cc:1047] running on GCE node
I20251024 08:15:07.617033 1217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:07.617326 1217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:07.618486 1217 hybrid_clock.cc:648] HybridClock initialized: now 1761293707618454 us; error 53 us; skew 500 ppm
I20251024 08:15:07.619858 1217 webserver.cc:492] Webserver started at http://127.31.115.193:36575/ using document root <none> and password file <none>
I20251024 08:15:07.620074 1217 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:07.620127 1217 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:07.620252 1217 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:07.621160 1217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "dd83f2d1389544eca10e6590c4e03e1c"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.621492 1217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "dd83f2d1389544eca10e6590c4e03e1c"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.622968 1217 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20251024 08:15:07.623903 1231 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:07.624109 1217 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:07.624192 1217 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "dd83f2d1389544eca10e6590c4e03e1c"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.624267 1217 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:07.635571 1217 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:07.635879 1217 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:07.636006 1217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:07.636258 1217 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:07.636683 1217 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:07.636722 1217 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:07.636787 1217 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:07.636818 1217 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:07.643373 1217 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.193:41517
I20251024 08:15:07.643507 1344 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.193:41517 every 8 connection(s)
I20251024 08:15:07.643805 1217 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20251024 08:15:07.649130 1345 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45705
I20251024 08:15:07.649272 1345 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:07.649487 1345 heartbeater.cc:507] Master 127.31.115.254:45705 requested a full tablet report, sending...
I20251024 08:15:07.650007 1157 ts_manager.cc:194] Registered new tserver with Master: dd83f2d1389544eca10e6590c4e03e1c (127.31.115.193:41517)
I20251024 08:15:07.650740 1157 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.193:48545
I20251024 08:15:07.652684 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 1217
I20251024 08:15:07.652776 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20251024 08:15:07.654245 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.194:0
--local_ip_for_outbound_sockets=127.31.115.194
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45705
--builtin_ntp_servers=127.31.115.212:45209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20251024 08:15:07.740737 1348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:07.740978 1348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:07.741012 1348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:07.742751 1348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:07.742851 1348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.194
I20251024 08:15:07.744757 1348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:45209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45705
--never_fsync=true
--heap_profile_path=/tmp/kudu.1348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.194
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:07.745080 1348 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:07.745361 1348 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:07.749228 1354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:07.749228 1356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:07.749488 1348 server_base.cc:1047] running on GCE node
W20251024 08:15:07.749454 1353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:07.749965 1348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:07.750743 1348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:07.751932 1348 hybrid_clock.cc:648] HybridClock initialized: now 1761293707751830 us; error 131 us; skew 500 ppm
I20251024 08:15:07.753629 1348 webserver.cc:492] Webserver started at http://127.31.115.194:44777/ using document root <none> and password file <none>
I20251024 08:15:07.754043 1348 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:07.754108 1348 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:07.754843 1348 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:07.755923 1348 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "f941904c2410425cae6fc2c154ec55fe"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.756265 1348 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "f941904c2410425cae6fc2c154ec55fe"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.757642 1348 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20251024 08:15:07.758482 1362 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:07.758759 1348 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:07.758849 1348 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "f941904c2410425cae6fc2c154ec55fe"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.758929 1348 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:07.767874 1348 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:07.768179 1348 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:07.768319 1348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:07.768570 1348 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:07.768939 1348 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:07.768980 1348 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:07.769018 1348 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:07.769038 1348 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:07.776651 1348 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.194:43683
I20251024 08:15:07.776710 1475 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.194:43683 every 8 connection(s)
I20251024 08:15:07.777110 1348 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20251024 08:15:07.779891 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 1348
I20251024 08:15:07.780019 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20251024 08:15:07.781311 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.195:0
--local_ip_for_outbound_sockets=127.31.115.195
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45705
--builtin_ntp_servers=127.31.115.212:45209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20251024 08:15:07.782949 1476 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45705
I20251024 08:15:07.783119 1476 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:07.783368 1476 heartbeater.cc:507] Master 127.31.115.254:45705 requested a full tablet report, sending...
I20251024 08:15:07.783797 1157 ts_manager.cc:194] Registered new tserver with Master: f941904c2410425cae6fc2c154ec55fe (127.31.115.194:43683)
I20251024 08:15:07.784364 1157 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.194:48003
W20251024 08:15:07.869539 1479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:07.869825 1479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:07.869853 1479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:07.871713 1479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:07.871799 1479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.195
I20251024 08:15:07.873595 1479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:45209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45705
--never_fsync=true
--heap_profile_path=/tmp/kudu.1479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.195
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:07.873900 1479 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:07.874187 1479 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:07.877413 1487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:07.877413 1485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:07.877413 1484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:07.878216 1479 server_base.cc:1047] running on GCE node
I20251024 08:15:07.878443 1479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:07.878736 1479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:07.879920 1479 hybrid_clock.cc:648] HybridClock initialized: now 1761293707879897 us; error 37 us; skew 500 ppm
I20251024 08:15:07.881479 1479 webserver.cc:492] Webserver started at http://127.31.115.195:44401/ using document root <none> and password file <none>
I20251024 08:15:07.881717 1479 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:07.881772 1479 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:07.881911 1479 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:07.882964 1479 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "b54fc25a6d624cb7865c41636c6322a3"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.883324 1479 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "b54fc25a6d624cb7865c41636c6322a3"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.884688 1479 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:07.885579 1493 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:07.885767 1479 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:07.885856 1479 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "b54fc25a6d624cb7865c41636c6322a3"
format_stamp: "Formatted at 2025-10-24 08:15:07 on dist-test-slave-c0j3"
I20251024 08:15:07.885932 1479 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:07.898278 1479 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:07.898707 1479 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:07.898851 1479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:07.899156 1479 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:07.899569 1479 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:07.899605 1479 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:07.899647 1479 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:07.899674 1479 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:07.907297 1479 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.195:36783
I20251024 08:15:07.907435 1606 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.195:36783 every 8 connection(s)
I20251024 08:15:07.907765 1479 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20251024 08:15:07.913633 1607 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45705
I20251024 08:15:07.913787 1607 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:07.914043 1607 heartbeater.cc:507] Master 127.31.115.254:45705 requested a full tablet report, sending...
I20251024 08:15:07.914494 1157 ts_manager.cc:194] Registered new tserver with Master: b54fc25a6d624cb7865c41636c6322a3 (127.31.115.195:36783)
I20251024 08:15:07.914958 1157 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.195:38039
I20251024 08:15:07.917726 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 1479
I20251024 08:15:07.917825 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20251024 08:15:07.919150 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:07.925639 1157 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42396:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20251024 08:15:07.927121 1157 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251024 08:15:07.935072 1541 tablet_service.cc:1505] Processing CreateTablet for tablet 0fce11298bfa413e98f4da04d3eb0e25 (DEFAULT_TABLE table=TestTable [id=61e37fd4e4614248a68193b5b056d079]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:07.935114 1279 tablet_service.cc:1505] Processing CreateTablet for tablet 0fce11298bfa413e98f4da04d3eb0e25 (DEFAULT_TABLE table=TestTable [id=61e37fd4e4614248a68193b5b056d079]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:07.935452 1541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fce11298bfa413e98f4da04d3eb0e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:07.935456 1279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fce11298bfa413e98f4da04d3eb0e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:07.935546 1410 tablet_service.cc:1505] Processing CreateTablet for tablet 0fce11298bfa413e98f4da04d3eb0e25 (DEFAULT_TABLE table=TestTable [id=61e37fd4e4614248a68193b5b056d079]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:07.935799 1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fce11298bfa413e98f4da04d3eb0e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:07.937786 1621 tablet_bootstrap.cc:492] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3: Bootstrap starting.
I20251024 08:15:07.938328 1622 tablet_bootstrap.cc:492] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c: Bootstrap starting.
I20251024 08:15:07.938570 1621 tablet_bootstrap.cc:654] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:07.938742 1623 tablet_bootstrap.cc:492] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe: Bootstrap starting.
I20251024 08:15:07.938915 1621 log.cc:826] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:07.939047 1622 tablet_bootstrap.cc:654] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:07.939374 1622 log.cc:826] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:07.939388 1623 tablet_bootstrap.cc:654] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:07.939662 1621 tablet_bootstrap.cc:492] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3: No bootstrap required, opened a new log
I20251024 08:15:07.939716 1623 log.cc:826] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:07.939735 1621 ts_tablet_manager.cc:1403] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251024 08:15:07.940109 1622 tablet_bootstrap.cc:492] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c: No bootstrap required, opened a new log
I20251024 08:15:07.940232 1622 ts_tablet_manager.cc:1403] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251024 08:15:07.940331 1623 tablet_bootstrap.cc:492] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe: No bootstrap required, opened a new log
I20251024 08:15:07.940404 1623 ts_tablet_manager.cc:1403] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:07.941546 1623 raft_consensus.cc:359] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } }
I20251024 08:15:07.941494 1621 raft_consensus.cc:359] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } }
I20251024 08:15:07.941674 1623 raft_consensus.cc:385] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:07.941676 1621 raft_consensus.cc:385] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:07.941695 1623 raft_consensus.cc:740] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f941904c2410425cae6fc2c154ec55fe, State: Initialized, Role: FOLLOWER
I20251024 08:15:07.941711 1621 raft_consensus.cc:740] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54fc25a6d624cb7865c41636c6322a3, State: Initialized, Role: FOLLOWER
I20251024 08:15:07.941821 1623 consensus_queue.cc:260] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } }
I20251024 08:15:07.941829 1621 consensus_queue.cc:260] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } }
I20251024 08:15:07.942121 1623 ts_tablet_manager.cc:1434] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251024 08:15:07.942121 1621 ts_tablet_manager.cc:1434] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20251024 08:15:07.942258 1476 heartbeater.cc:499] Master 127.31.115.254:45705 was elected leader, sending a full tablet report...
I20251024 08:15:07.942309 1607 heartbeater.cc:499] Master 127.31.115.254:45705 was elected leader, sending a full tablet report...
I20251024 08:15:07.942152 1622 raft_consensus.cc:359] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } }
I20251024 08:15:07.942400 1622 raft_consensus.cc:385] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:07.942464 1622 raft_consensus.cc:740] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd83f2d1389544eca10e6590c4e03e1c, State: Initialized, Role: FOLLOWER
I20251024 08:15:07.942612 1622 consensus_queue.cc:260] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } }
I20251024 08:15:07.942926 1622 ts_tablet_manager.cc:1434] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251024 08:15:07.943065 1345 heartbeater.cc:499] Master 127.31.115.254:45705 was elected leader, sending a full tablet report...
I20251024 08:15:07.946677 1627 raft_consensus.cc:493] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251024 08:15:07.946817 1627 raft_consensus.cc:515] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } }
I20251024 08:15:07.947186 1627 leader_election.cc:290] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f941904c2410425cae6fc2c154ec55fe (127.31.115.194:43683), dd83f2d1389544eca10e6590c4e03e1c (127.31.115.193:41517)
I20251024 08:15:07.950618 1430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0fce11298bfa413e98f4da04d3eb0e25" candidate_uuid: "b54fc25a6d624cb7865c41636c6322a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f941904c2410425cae6fc2c154ec55fe" is_pre_election: true
I20251024 08:15:07.950798 1430 raft_consensus.cc:2468] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54fc25a6d624cb7865c41636c6322a3 in term 0.
I20251024 08:15:07.951040 1495 leader_election.cc:304] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b54fc25a6d624cb7865c41636c6322a3, f941904c2410425cae6fc2c154ec55fe; no voters:
I20251024 08:15:07.951184 1627 raft_consensus.cc:2804] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251024 08:15:07.951254 1627 raft_consensus.cc:493] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251024 08:15:07.951288 1627 raft_consensus.cc:3060] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:07.951311 1299 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0fce11298bfa413e98f4da04d3eb0e25" candidate_uuid: "b54fc25a6d624cb7865c41636c6322a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd83f2d1389544eca10e6590c4e03e1c" is_pre_election: true
I20251024 08:15:07.951560 1299 raft_consensus.cc:2468] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54fc25a6d624cb7865c41636c6322a3 in term 0.
I20251024 08:15:07.952159 1627 raft_consensus.cc:515] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } }
I20251024 08:15:07.952325 1627 leader_election.cc:290] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [CANDIDATE]: Term 1 election: Requested vote from peers f941904c2410425cae6fc2c154ec55fe (127.31.115.194:43683), dd83f2d1389544eca10e6590c4e03e1c (127.31.115.193:41517)
I20251024 08:15:07.952584 1430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0fce11298bfa413e98f4da04d3eb0e25" candidate_uuid: "b54fc25a6d624cb7865c41636c6322a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f941904c2410425cae6fc2c154ec55fe"
I20251024 08:15:07.952631 1299 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0fce11298bfa413e98f4da04d3eb0e25" candidate_uuid: "b54fc25a6d624cb7865c41636c6322a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd83f2d1389544eca10e6590c4e03e1c"
I20251024 08:15:07.952695 1430 raft_consensus.cc:3060] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:07.952757 1299 raft_consensus.cc:3060] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:07.953418 1430 raft_consensus.cc:2468] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54fc25a6d624cb7865c41636c6322a3 in term 1.
I20251024 08:15:07.953657 1299 raft_consensus.cc:2468] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54fc25a6d624cb7865c41636c6322a3 in term 1.
I20251024 08:15:07.953660 1495 leader_election.cc:304] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b54fc25a6d624cb7865c41636c6322a3, f941904c2410425cae6fc2c154ec55fe; no voters:
I20251024 08:15:07.953873 1627 raft_consensus.cc:2804] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:07.954056 1627 raft_consensus.cc:697] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [term 1 LEADER]: Becoming Leader. State: Replica: b54fc25a6d624cb7865c41636c6322a3, State: Running, Role: LEADER
I20251024 08:15:07.954165 1627 consensus_queue.cc:237] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } }
I20251024 08:15:07.955089 1157 catalog_manager.cc:5649] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b54fc25a6d624cb7865c41636c6322a3 (127.31.115.195). New cstate: current_term: 1 leader_uuid: "b54fc25a6d624cb7865c41636c6322a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } health_report { overall_health: UNKNOWN } } }
I20251024 08:15:07.959892 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:07.960536 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver dd83f2d1389544eca10e6590c4e03e1c to finish bootstrapping
I20251024 08:15:07.964510 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver f941904c2410425cae6fc2c154ec55fe to finish bootstrapping
I20251024 08:15:07.968993 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver b54fc25a6d624cb7865c41636c6322a3 to finish bootstrapping
I20251024 08:15:07.974185 1157 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42396:
name: "TestAnotherTable"
schema {
columns {
name: "foo"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "bar"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
comment: "comment for bar"
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "foo"
}
}
}
W20251024 08:15:07.974449 1157 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestAnotherTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251024 08:15:07.976449 1279 tablet_service.cc:1505] Processing CreateTablet for tablet d94cabceeaa64587adb39259f6da0a43 (DEFAULT_TABLE table=TestAnotherTable [id=c887b96c30c2440cb267e47a73bc8c41]), partition=RANGE (foo) PARTITION UNBOUNDED
I20251024 08:15:07.976500 1541 tablet_service.cc:1505] Processing CreateTablet for tablet d94cabceeaa64587adb39259f6da0a43 (DEFAULT_TABLE table=TestAnotherTable [id=c887b96c30c2440cb267e47a73bc8c41]), partition=RANGE (foo) PARTITION UNBOUNDED
I20251024 08:15:07.976487 1410 tablet_service.cc:1505] Processing CreateTablet for tablet d94cabceeaa64587adb39259f6da0a43 (DEFAULT_TABLE table=TestAnotherTable [id=c887b96c30c2440cb267e47a73bc8c41]), partition=RANGE (foo) PARTITION UNBOUNDED
I20251024 08:15:07.976644 1541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94cabceeaa64587adb39259f6da0a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:07.976644 1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94cabceeaa64587adb39259f6da0a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:07.976645 1279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94cabceeaa64587adb39259f6da0a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:07.977710 1622 tablet_bootstrap.cc:492] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c: Bootstrap starting.
I20251024 08:15:07.977941 1623 tablet_bootstrap.cc:492] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe: Bootstrap starting.
I20251024 08:15:07.977959 1621 tablet_bootstrap.cc:492] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3: Bootstrap starting.
I20251024 08:15:07.978278 1622 tablet_bootstrap.cc:654] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:07.978397 1621 tablet_bootstrap.cc:654] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:07.978442 1623 tablet_bootstrap.cc:654] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:07.979173 1623 tablet_bootstrap.cc:492] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe: No bootstrap required, opened a new log
I20251024 08:15:07.979247 1623 ts_tablet_manager.cc:1403] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:07.979249 1621 tablet_bootstrap.cc:492] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3: No bootstrap required, opened a new log
I20251024 08:15:07.979274 1622 tablet_bootstrap.cc:492] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c: No bootstrap required, opened a new log
I20251024 08:15:07.979301 1621 ts_tablet_manager.cc:1403] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:07.979319 1622 ts_tablet_manager.cc:1403] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251024 08:15:07.979463 1623 raft_consensus.cc:359] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } }
I20251024 08:15:07.979506 1622 raft_consensus.cc:359] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } }
I20251024 08:15:07.979504 1621 raft_consensus.cc:359] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } }
I20251024 08:15:07.979560 1623 raft_consensus.cc:385] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:07.979578 1622 raft_consensus.cc:385] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:07.979583 1621 raft_consensus.cc:385] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:07.979586 1623 raft_consensus.cc:740] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f941904c2410425cae6fc2c154ec55fe, State: Initialized, Role: FOLLOWER
I20251024 08:15:07.979605 1621 raft_consensus.cc:740] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54fc25a6d624cb7865c41636c6322a3, State: Initialized, Role: FOLLOWER
I20251024 08:15:07.979604 1622 raft_consensus.cc:740] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd83f2d1389544eca10e6590c4e03e1c, State: Initialized, Role: FOLLOWER
I20251024 08:15:07.979661 1621 consensus_queue.cc:260] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } }
I20251024 08:15:07.979662 1622 consensus_queue.cc:260] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } }
I20251024 08:15:07.979664 1623 consensus_queue.cc:260] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } }
I20251024 08:15:07.979751 1621 ts_tablet_manager.cc:1434] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251024 08:15:07.979764 1622 ts_tablet_manager.cc:1434] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251024 08:15:07.979774 1623 ts_tablet_manager.cc:1434] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
W20251024 08:15:08.027858 1477 tablet.cc:2378] T 0fce11298bfa413e98f4da04d3eb0e25 P f941904c2410425cae6fc2c154ec55fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251024 08:15:08.118350 1627 raft_consensus.cc:493] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251024 08:15:08.118467 1627 raft_consensus.cc:515] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } }
I20251024 08:15:08.118649 1627 leader_election.cc:290] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd83f2d1389544eca10e6590c4e03e1c (127.31.115.193:41517), f941904c2410425cae6fc2c154ec55fe (127.31.115.194:43683)
I20251024 08:15:08.118995 1299 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d94cabceeaa64587adb39259f6da0a43" candidate_uuid: "b54fc25a6d624cb7865c41636c6322a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd83f2d1389544eca10e6590c4e03e1c" is_pre_election: true
I20251024 08:15:08.119071 1430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d94cabceeaa64587adb39259f6da0a43" candidate_uuid: "b54fc25a6d624cb7865c41636c6322a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f941904c2410425cae6fc2c154ec55fe" is_pre_election: true
I20251024 08:15:08.119164 1430 raft_consensus.cc:2468] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54fc25a6d624cb7865c41636c6322a3 in term 0.
I20251024 08:15:08.119163 1299 raft_consensus.cc:2468] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54fc25a6d624cb7865c41636c6322a3 in term 0.
I20251024 08:15:08.119418 1495 leader_election.cc:304] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b54fc25a6d624cb7865c41636c6322a3, dd83f2d1389544eca10e6590c4e03e1c; no voters:
I20251024 08:15:08.119535 1627 raft_consensus.cc:2804] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251024 08:15:08.119575 1627 raft_consensus.cc:493] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251024 08:15:08.119598 1627 raft_consensus.cc:3060] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:08.120205 1627 raft_consensus.cc:515] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } }
I20251024 08:15:08.120337 1627 leader_election.cc:290] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [CANDIDATE]: Term 1 election: Requested vote from peers dd83f2d1389544eca10e6590c4e03e1c (127.31.115.193:41517), f941904c2410425cae6fc2c154ec55fe (127.31.115.194:43683)
I20251024 08:15:08.120664 1430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d94cabceeaa64587adb39259f6da0a43" candidate_uuid: "b54fc25a6d624cb7865c41636c6322a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f941904c2410425cae6fc2c154ec55fe"
I20251024 08:15:08.120757 1430 raft_consensus.cc:3060] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:08.120709 1299 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d94cabceeaa64587adb39259f6da0a43" candidate_uuid: "b54fc25a6d624cb7865c41636c6322a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd83f2d1389544eca10e6590c4e03e1c"
I20251024 08:15:08.120808 1299 raft_consensus.cc:3060] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:08.121560 1430 raft_consensus.cc:2468] T d94cabceeaa64587adb39259f6da0a43 P f941904c2410425cae6fc2c154ec55fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54fc25a6d624cb7865c41636c6322a3 in term 1.
I20251024 08:15:08.121560 1299 raft_consensus.cc:2468] T d94cabceeaa64587adb39259f6da0a43 P dd83f2d1389544eca10e6590c4e03e1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54fc25a6d624cb7865c41636c6322a3 in term 1.
I20251024 08:15:08.121819 1495 leader_election.cc:304] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b54fc25a6d624cb7865c41636c6322a3, dd83f2d1389544eca10e6590c4e03e1c; no voters:
I20251024 08:15:08.121942 1627 raft_consensus.cc:2804] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:08.122004 1627 raft_consensus.cc:697] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [term 1 LEADER]: Becoming Leader. State: Replica: b54fc25a6d624cb7865c41636c6322a3, State: Running, Role: LEADER
I20251024 08:15:08.122049 1627 consensus_queue.cc:237] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } } peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } }
I20251024 08:15:08.122807 1154 catalog_manager.cc:5649] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b54fc25a6d624cb7865c41636c6322a3 (127.31.115.195). New cstate: current_term: 1 leader_uuid: "b54fc25a6d624cb7865c41636c6322a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b54fc25a6d624cb7865c41636c6322a3" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36783 } health_report { overall_health: HEALTHY } } }
W20251024 08:15:08.144811 1346 tablet.cc:2378] T 0fce11298bfa413e98f4da04d3eb0e25 P dd83f2d1389544eca10e6590c4e03e1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20251024 08:15:08.158421 1608 tablet.cc:2378] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20251024 08:15:08.223985 1637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:08.224298 1637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:08.226563 1637 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20251024 08:15:08.391817 1647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:08.392050 1647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:08.393705 1647 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20251024 08:15:08.518200 1627 consensus_queue.cc:1048] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:08.520802 1627 consensus_queue.cc:1048] T d94cabceeaa64587adb39259f6da0a43 P b54fc25a6d624cb7865c41636c6322a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251024 08:15:08.549939 1657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:08.550144 1657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:08.551787 1657 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20251024 08:15:08.573845 1627 consensus_queue.cc:1048] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f941904c2410425cae6fc2c154ec55fe" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 43683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:08.575851 1627 consensus_queue.cc:1048] T 0fce11298bfa413e98f4da04d3eb0e25 P b54fc25a6d624cb7865c41636c6322a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd83f2d1389544eca10e6590c4e03e1c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 41517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251024 08:15:08.712311 1676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:08.712565 1676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:08.714301 1676 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20251024 08:15:08.786404 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 1217
I20251024 08:15:08.792955 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 1348
I20251024 08:15:08.799350 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 1479
I20251024 08:15:08.805081 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 1125
2025-10-24T08:15:08Z chronyd exiting
[ OK ] AdminCliTest.TestDescribeTableColumnFlags (1491 ms)
[ RUN ] AdminCliTest.TestAuthzResetCacheNotAuthorized
I20251024 08:15:08.826771 32207 test_util.cc:276] Using random seed: 635181194
I20251024 08:15:08.827396 32207 ts_itest-base.cc:115] Starting cluster with:
I20251024 08:15:08.827440 32207 ts_itest-base.cc:116] --------------
I20251024 08:15:08.827451 32207 ts_itest-base.cc:117] 3 tablet servers
I20251024 08:15:08.827462 32207 ts_itest-base.cc:118] 3 replicas per TS
I20251024 08:15:08.827476 32207 ts_itest-base.cc:119] --------------
2025-10-24T08:15:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-24T08:15:08Z Disabled control of system clock
I20251024 08:15:08.835886 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:46513
--webserver_interface=127.31.115.254
--webserver_port=0
--builtin_ntp_servers=127.31.115.212:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:46513
--superuser_acl=no-such-user with env {}
W20251024 08:15:08.922354 1693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:08.922612 1693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:08.922652 1693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:08.924284 1693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:08.924360 1693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:08.924382 1693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:08.924405 1693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:08.926236 1693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:46513
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:46513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.1693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:08.926615 1693 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:08.926914 1693 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:08.929783 1701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:08.929786 1698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:08.929786 1699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:08.931201 1693 server_base.cc:1047] running on GCE node
I20251024 08:15:08.931555 1693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:08.931869 1693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:08.933034 1693 hybrid_clock.cc:648] HybridClock initialized: now 1761293708933011 us; error 56 us; skew 500 ppm
I20251024 08:15:08.934588 1693 webserver.cc:492] Webserver started at http://127.31.115.254:42763/ using document root <none> and password file <none>
I20251024 08:15:08.934847 1693 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:08.934903 1693 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:08.935036 1693 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:08.935988 1693 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "fca26fce078f4b8292270a0ef125957f"
format_stamp: "Formatted at 2025-10-24 08:15:08 on dist-test-slave-c0j3"
I20251024 08:15:08.936331 1693 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "fca26fce078f4b8292270a0ef125957f"
format_stamp: "Formatted at 2025-10-24 08:15:08 on dist-test-slave-c0j3"
I20251024 08:15:08.937748 1693 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20251024 08:15:08.938650 1707 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:08.938880 1693 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:08.938970 1693 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
uuid: "fca26fce078f4b8292270a0ef125957f"
format_stamp: "Formatted at 2025-10-24 08:15:08 on dist-test-slave-c0j3"
I20251024 08:15:08.939049 1693 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:08.950337 1693 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:08.950713 1693 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:08.950839 1693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:08.955101 1693 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:46513
I20251024 08:15:08.955137 1759 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:46513 every 8 connection(s)
I20251024 08:15:08.955526 1693 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20251024 08:15:08.956189 1760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:08.958711 1760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f: Bootstrap starting.
I20251024 08:15:08.959344 1760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:08.959595 1760 log.cc:826] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:08.960266 1760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f: No bootstrap required, opened a new log
I20251024 08:15:08.961338 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 1693
I20251024 08:15:08.961438 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance
I20251024 08:15:08.961627 1760 raft_consensus.cc:359] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fca26fce078f4b8292270a0ef125957f" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 46513 } }
I20251024 08:15:08.961778 1760 raft_consensus.cc:385] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:08.961814 1760 raft_consensus.cc:740] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fca26fce078f4b8292270a0ef125957f, State: Initialized, Role: FOLLOWER
I20251024 08:15:08.961963 1760 consensus_queue.cc:260] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fca26fce078f4b8292270a0ef125957f" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 46513 } }
I20251024 08:15:08.962045 1760 raft_consensus.cc:399] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:08.962080 1760 raft_consensus.cc:493] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:08.962129 1760 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:08.962884 1760 raft_consensus.cc:515] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fca26fce078f4b8292270a0ef125957f" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 46513 } }
I20251024 08:15:08.963017 1760 leader_election.cc:304] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fca26fce078f4b8292270a0ef125957f; no voters:
I20251024 08:15:08.963210 1760 leader_election.cc:290] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:08.963258 1765 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:08.963423 1765 raft_consensus.cc:697] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [term 1 LEADER]: Becoming Leader. State: Replica: fca26fce078f4b8292270a0ef125957f, State: Running, Role: LEADER
I20251024 08:15:08.963505 1760 sys_catalog.cc:565] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:08.963537 1765 consensus_queue.cc:237] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fca26fce078f4b8292270a0ef125957f" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 46513 } }
I20251024 08:15:08.963846 1765 sys_catalog.cc:455] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fca26fce078f4b8292270a0ef125957f. Latest consensus state: current_term: 1 leader_uuid: "fca26fce078f4b8292270a0ef125957f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fca26fce078f4b8292270a0ef125957f" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 46513 } } }
I20251024 08:15:08.963858 1767 sys_catalog.cc:455] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fca26fce078f4b8292270a0ef125957f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fca26fce078f4b8292270a0ef125957f" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 46513 } } }
I20251024 08:15:08.963938 1767 sys_catalog.cc:458] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:08.963937 1765 sys_catalog.cc:458] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:08.964172 1770 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:08.964716 1770 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:08.966444 1770 catalog_manager.cc:1357] Generated new cluster ID: 08ccf8d7f655497d9d02ba8cbb2659cc
I20251024 08:15:08.966540 1770 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:08.978083 1770 catalog_manager.cc:1380] Generated new certificate authority record
I20251024 08:15:08.978999 1770 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:08.984334 1770 catalog_manager.cc:6022] T 00000000000000000000000000000000 P fca26fce078f4b8292270a0ef125957f: Generated new TSK 0
I20251024 08:15:08.984583 1770 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251024 08:15:08.988020 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.193:0
--local_ip_for_outbound_sockets=127.31.115.193
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:46513
--builtin_ntp_servers=127.31.115.212:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20251024 08:15:09.096294 1784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:09.096513 1784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:09.096545 1784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:09.098237 1784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:09.098346 1784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.193
I20251024 08:15:09.100332 1784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:46513
--never_fsync=true
--heap_profile_path=/tmp/kudu.1784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.193
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:09.100670 1784 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:09.100931 1784 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:09.103837 1789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:09.103834 1790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:09.104022 1784 server_base.cc:1047] running on GCE node
W20251024 08:15:09.104156 1792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:09.104440 1784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:09.104699 1784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:09.105964 1784 hybrid_clock.cc:648] HybridClock initialized: now 1761293709105923 us; error 40 us; skew 500 ppm
I20251024 08:15:09.107906 1784 webserver.cc:492] Webserver started at http://127.31.115.193:46593/ using document root <none> and password file <none>
I20251024 08:15:09.108201 1784 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:09.108253 1784 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:09.108383 1784 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:09.109467 1784 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "a817c8ead3334f9e83210475da830972"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.109812 1784 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "a817c8ead3334f9e83210475da830972"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.111274 1784 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20251024 08:15:09.112154 1798 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.112370 1784 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:09.112447 1784 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "a817c8ead3334f9e83210475da830972"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.112524 1784 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:09.124091 1784 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:09.124409 1784 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:09.124560 1784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:09.124809 1784 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:09.125185 1784 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:09.125221 1784 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.125247 1784 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:09.125262 1784 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.132910 1784 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.193:46591
I20251024 08:15:09.132987 1911 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.193:46591 every 8 connection(s)
I20251024 08:15:09.133324 1784 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20251024 08:15:09.134714 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 1784
I20251024 08:15:09.134831 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20251024 08:15:09.135993 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.194:0
--local_ip_for_outbound_sockets=127.31.115.194
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:46513
--builtin_ntp_servers=127.31.115.212:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20251024 08:15:09.139576 1912 heartbeater.cc:344] Connected to a master server at 127.31.115.254:46513
I20251024 08:15:09.139735 1912 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:09.140029 1912 heartbeater.cc:507] Master 127.31.115.254:46513 requested a full tablet report, sending...
I20251024 08:15:09.140676 1724 ts_manager.cc:194] Registered new tserver with Master: a817c8ead3334f9e83210475da830972 (127.31.115.193:46591)
I20251024 08:15:09.141772 1724 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.193:51479
W20251024 08:15:09.221760 1915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:09.221977 1915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:09.222014 1915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:09.223711 1915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:09.223807 1915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.194
I20251024 08:15:09.225649 1915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:46513
--never_fsync=true
--heap_profile_path=/tmp/kudu.1915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.194
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:09.225988 1915 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:09.226256 1915 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:09.229256 1923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:09.229256 1921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:09.229372 1915 server_base.cc:1047] running on GCE node
W20251024 08:15:09.229256 1920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:09.229665 1915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:09.229907 1915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:09.231087 1915 hybrid_clock.cc:648] HybridClock initialized: now 1761293709231063 us; error 43 us; skew 500 ppm
I20251024 08:15:09.232626 1915 webserver.cc:492] Webserver started at http://127.31.115.194:36551/ using document root <none> and password file <none>
I20251024 08:15:09.232900 1915 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:09.232954 1915 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:09.233086 1915 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:09.234046 1915 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "e376ab28b4454aafa8838a6999b5bf0e"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.234390 1915 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "e376ab28b4454aafa8838a6999b5bf0e"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.235879 1915 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:09.236790 1929 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.237020 1915 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:09.237112 1915 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "e376ab28b4454aafa8838a6999b5bf0e"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.237192 1915 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:09.252646 1915 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:09.252943 1915 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:09.253062 1915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:09.253289 1915 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:09.253639 1915 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:09.253674 1915 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.253701 1915 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:09.253732 1915 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.259701 1915 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.194:42345
I20251024 08:15:09.259792 2042 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.194:42345 every 8 connection(s)
I20251024 08:15:09.260073 1915 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20251024 08:15:09.261659 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 1915
I20251024 08:15:09.261754 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20251024 08:15:09.263021 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.195:0
--local_ip_for_outbound_sockets=127.31.115.195
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:46513
--builtin_ntp_servers=127.31.115.212:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20251024 08:15:09.265656 2043 heartbeater.cc:344] Connected to a master server at 127.31.115.254:46513
I20251024 08:15:09.265789 2043 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:09.266036 2043 heartbeater.cc:507] Master 127.31.115.254:46513 requested a full tablet report, sending...
I20251024 08:15:09.266496 1724 ts_manager.cc:194] Registered new tserver with Master: e376ab28b4454aafa8838a6999b5bf0e (127.31.115.194:42345)
I20251024 08:15:09.266947 1724 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.194:43643
W20251024 08:15:09.355432 2046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:09.355654 2046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:09.355677 2046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:09.357352 2046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:09.357450 2046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.195
I20251024 08:15:09.359421 2046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:46513
--never_fsync=true
--heap_profile_path=/tmp/kudu.2046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.195
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:09.359745 2046 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:09.360011 2046 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:09.362886 2052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:09.362843 2051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:09.362859 2054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:09.363015 2046 server_base.cc:1047] running on GCE node
I20251024 08:15:09.363240 2046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:09.363469 2046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:09.364631 2046 hybrid_clock.cc:648] HybridClock initialized: now 1761293709364622 us; error 42 us; skew 500 ppm
I20251024 08:15:09.366050 2046 webserver.cc:492] Webserver started at http://127.31.115.195:43569/ using document root <none> and password file <none>
I20251024 08:15:09.366302 2046 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:09.366358 2046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:09.366487 2046 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:09.367501 2046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "65dce859a04548f3bf8cfecc41537241"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.367838 2046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "65dce859a04548f3bf8cfecc41537241"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.369293 2046 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20251024 08:15:09.370160 2060 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.370414 2046 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:09.370534 2046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "65dce859a04548f3bf8cfecc41537241"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.370610 2046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:09.391244 2046 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:09.391567 2046 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:09.391707 2046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:09.391973 2046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:09.392354 2046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:09.392390 2046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.392431 2046 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:09.392452 2046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.399156 2046 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.195:38015
I20251024 08:15:09.399231 2173 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.195:38015 every 8 connection(s)
I20251024 08:15:09.399576 2046 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20251024 08:15:09.404521 2174 heartbeater.cc:344] Connected to a master server at 127.31.115.254:46513
I20251024 08:15:09.404665 2174 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:09.404913 2174 heartbeater.cc:507] Master 127.31.115.254:46513 requested a full tablet report, sending...
I20251024 08:15:09.405364 1724 ts_manager.cc:194] Registered new tserver with Master: 65dce859a04548f3bf8cfecc41537241 (127.31.115.195:38015)
I20251024 08:15:09.405771 1724 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.195:56909
I20251024 08:15:09.409236 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 2046
I20251024 08:15:09.409341 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20251024 08:15:09.410841 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:09.417680 1724 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42388:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20251024 08:15:09.419229 1724 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251024 08:15:09.427160 2108 tablet_service.cc:1505] Processing CreateTablet for tablet e296ea373c1c488d859e60c45822465f (DEFAULT_TABLE table=TestTable [id=a3225b6a35564b278bbb3f9c4c4abc76]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:09.427588 2108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e296ea373c1c488d859e60c45822465f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:09.427752 1846 tablet_service.cc:1505] Processing CreateTablet for tablet e296ea373c1c488d859e60c45822465f (DEFAULT_TABLE table=TestTable [id=a3225b6a35564b278bbb3f9c4c4abc76]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:09.428108 1846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e296ea373c1c488d859e60c45822465f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:09.430210 2188 tablet_bootstrap.cc:492] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241: Bootstrap starting.
I20251024 08:15:09.430750 2189 tablet_bootstrap.cc:492] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972: Bootstrap starting.
I20251024 08:15:09.431066 2188 tablet_bootstrap.cc:654] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:09.431413 2189 tablet_bootstrap.cc:654] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:09.431726 2189 log.cc:826] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:09.431790 1976 tablet_service.cc:1505] Processing CreateTablet for tablet e296ea373c1c488d859e60c45822465f (DEFAULT_TABLE table=TestTable [id=a3225b6a35564b278bbb3f9c4c4abc76]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:09.432107 1976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e296ea373c1c488d859e60c45822465f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:09.432520 2189 tablet_bootstrap.cc:492] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972: No bootstrap required, opened a new log
I20251024 08:15:09.432598 2189 ts_tablet_manager.cc:1403] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:09.434366 2189 raft_consensus.cc:359] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e376ab28b4454aafa8838a6999b5bf0e" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 42345 } } peers { permanent_uuid: "65dce859a04548f3bf8cfecc41537241" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 38015 } } peers { permanent_uuid: "a817c8ead3334f9e83210475da830972" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46591 } }
I20251024 08:15:09.434603 2189 raft_consensus.cc:385] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:09.434643 2189 raft_consensus.cc:740] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a817c8ead3334f9e83210475da830972, State: Initialized, Role: FOLLOWER
I20251024 08:15:09.434813 2189 consensus_queue.cc:260] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e376ab28b4454aafa8838a6999b5bf0e" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 42345 } } peers { permanent_uuid: "65dce859a04548f3bf8cfecc41537241" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 38015 } } peers { permanent_uuid: "a817c8ead3334f9e83210475da830972" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46591 } }
I20251024 08:15:09.435148 2189 ts_tablet_manager.cc:1434] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251024 08:15:09.435317 1912 heartbeater.cc:499] Master 127.31.115.254:46513 was elected leader, sending a full tablet report...
I20251024 08:15:09.435408 2191 tablet_bootstrap.cc:492] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e: Bootstrap starting.
I20251024 08:15:09.435573 2188 log.cc:826] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:09.436370 2188 tablet_bootstrap.cc:492] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241: No bootstrap required, opened a new log
I20251024 08:15:09.436455 2191 tablet_bootstrap.cc:654] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:09.436476 2188 ts_tablet_manager.cc:1403] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251024 08:15:09.436797 2191 log.cc:826] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:09.437778 2191 tablet_bootstrap.cc:492] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e: No bootstrap required, opened a new log
I20251024 08:15:09.437857 2191 ts_tablet_manager.cc:1403] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251024 08:15:09.438099 2188 raft_consensus.cc:359] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e376ab28b4454aafa8838a6999b5bf0e" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 42345 } } peers { permanent_uuid: "65dce859a04548f3bf8cfecc41537241" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 38015 } } peers { permanent_uuid: "a817c8ead3334f9e83210475da830972" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46591 } }
I20251024 08:15:09.438261 2188 raft_consensus.cc:385] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:09.438280 2188 raft_consensus.cc:740] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dce859a04548f3bf8cfecc41537241, State: Initialized, Role: FOLLOWER
I20251024 08:15:09.438383 2188 consensus_queue.cc:260] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e376ab28b4454aafa8838a6999b5bf0e" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 42345 } } peers { permanent_uuid: "65dce859a04548f3bf8cfecc41537241" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 38015 } } peers { permanent_uuid: "a817c8ead3334f9e83210475da830972" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46591 } }
I20251024 08:15:09.439112 2188 ts_tablet_manager.cc:1434] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251024 08:15:09.439215 2174 heartbeater.cc:499] Master 127.31.115.254:46513 was elected leader, sending a full tablet report...
I20251024 08:15:09.440227 2191 raft_consensus.cc:359] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e376ab28b4454aafa8838a6999b5bf0e" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 42345 } } peers { permanent_uuid: "65dce859a04548f3bf8cfecc41537241" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 38015 } } peers { permanent_uuid: "a817c8ead3334f9e83210475da830972" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46591 } }
I20251024 08:15:09.440430 2191 raft_consensus.cc:385] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:09.440485 2191 raft_consensus.cc:740] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e376ab28b4454aafa8838a6999b5bf0e, State: Initialized, Role: FOLLOWER
I20251024 08:15:09.440596 2191 consensus_queue.cc:260] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e376ab28b4454aafa8838a6999b5bf0e" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 42345 } } peers { permanent_uuid: "65dce859a04548f3bf8cfecc41537241" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 38015 } } peers { permanent_uuid: "a817c8ead3334f9e83210475da830972" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46591 } }
I20251024 08:15:09.440920 2191 ts_tablet_manager.cc:1434] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251024 08:15:09.440968 2043 heartbeater.cc:499] Master 127.31.115.254:46513 was elected leader, sending a full tablet report...
I20251024 08:15:09.473088 2192 raft_consensus.cc:493] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251024 08:15:09.473306 2192 raft_consensus.cc:515] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e376ab28b4454aafa8838a6999b5bf0e" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 42345 } } peers { permanent_uuid: "65dce859a04548f3bf8cfecc41537241" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 38015 } } peers { permanent_uuid: "a817c8ead3334f9e83210475da830972" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46591 } }
I20251024 08:15:09.473737 2192 leader_election.cc:290] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e376ab28b4454aafa8838a6999b5bf0e (127.31.115.194:42345), 65dce859a04548f3bf8cfecc41537241 (127.31.115.195:38015)
I20251024 08:15:09.476785 1997 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e296ea373c1c488d859e60c45822465f" candidate_uuid: "a817c8ead3334f9e83210475da830972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e376ab28b4454aafa8838a6999b5bf0e" is_pre_election: true
I20251024 08:15:09.476995 1997 raft_consensus.cc:2468] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a817c8ead3334f9e83210475da830972 in term 0.
I20251024 08:15:09.477273 2128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e296ea373c1c488d859e60c45822465f" candidate_uuid: "a817c8ead3334f9e83210475da830972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dce859a04548f3bf8cfecc41537241" is_pre_election: true
I20251024 08:15:09.477412 2128 raft_consensus.cc:2468] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a817c8ead3334f9e83210475da830972 in term 0.
I20251024 08:15:09.477455 1799 leader_election.cc:304] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a817c8ead3334f9e83210475da830972, e376ab28b4454aafa8838a6999b5bf0e; no voters:
I20251024 08:15:09.477653 2192 raft_consensus.cc:2804] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251024 08:15:09.477734 2192 raft_consensus.cc:493] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251024 08:15:09.477756 2192 raft_consensus.cc:3060] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:09.478453 2192 raft_consensus.cc:515] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e376ab28b4454aafa8838a6999b5bf0e" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 42345 } } peers { permanent_uuid: "65dce859a04548f3bf8cfecc41537241" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 38015 } } peers { permanent_uuid: "a817c8ead3334f9e83210475da830972" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46591 } }
I20251024 08:15:09.478647 2192 leader_election.cc:290] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [CANDIDATE]: Term 1 election: Requested vote from peers e376ab28b4454aafa8838a6999b5bf0e (127.31.115.194:42345), 65dce859a04548f3bf8cfecc41537241 (127.31.115.195:38015)
I20251024 08:15:09.478857 2128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e296ea373c1c488d859e60c45822465f" candidate_uuid: "a817c8ead3334f9e83210475da830972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dce859a04548f3bf8cfecc41537241"
I20251024 08:15:09.478857 1997 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e296ea373c1c488d859e60c45822465f" candidate_uuid: "a817c8ead3334f9e83210475da830972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e376ab28b4454aafa8838a6999b5bf0e"
I20251024 08:15:09.478950 1997 raft_consensus.cc:3060] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:09.478950 2128 raft_consensus.cc:3060] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:09.479846 2128 raft_consensus.cc:2468] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a817c8ead3334f9e83210475da830972 in term 1.
I20251024 08:15:09.479846 1997 raft_consensus.cc:2468] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a817c8ead3334f9e83210475da830972 in term 1.
I20251024 08:15:09.480085 1800 leader_election.cc:304] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dce859a04548f3bf8cfecc41537241, a817c8ead3334f9e83210475da830972; no voters:
I20251024 08:15:09.480206 2192 raft_consensus.cc:2804] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:09.480357 2192 raft_consensus.cc:697] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [term 1 LEADER]: Becoming Leader. State: Replica: a817c8ead3334f9e83210475da830972, State: Running, Role: LEADER
I20251024 08:15:09.480466 2192 consensus_queue.cc:237] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e376ab28b4454aafa8838a6999b5bf0e" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 42345 } } peers { permanent_uuid: "65dce859a04548f3bf8cfecc41537241" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 38015 } } peers { permanent_uuid: "a817c8ead3334f9e83210475da830972" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46591 } }
I20251024 08:15:09.481362 1723 catalog_manager.cc:5649] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972 reported cstate change: term changed from 0 to 1, leader changed from <none> to a817c8ead3334f9e83210475da830972 (127.31.115.193). New cstate: current_term: 1 leader_uuid: "a817c8ead3334f9e83210475da830972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e376ab28b4454aafa8838a6999b5bf0e" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 42345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dce859a04548f3bf8cfecc41537241" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 38015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a817c8ead3334f9e83210475da830972" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46591 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:09.485137 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:09.485634 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver a817c8ead3334f9e83210475da830972 to finish bootstrapping
I20251024 08:15:09.489733 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e376ab28b4454aafa8838a6999b5bf0e to finish bootstrapping
I20251024 08:15:09.494774 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 65dce859a04548f3bf8cfecc41537241 to finish bootstrapping
W20251024 08:15:09.510694 2044 tablet.cc:2378] T e296ea373c1c488d859e60c45822465f P e376ab28b4454aafa8838a6999b5bf0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20251024 08:15:09.603744 1723 server_base.cc:1129] Unauthorized access attempt to method kudu.master.MasterService.RefreshAuthzCache from {username='slave'} at 127.0.0.1:42400
W20251024 08:15:09.634178 1913 tablet.cc:2378] T e296ea373c1c488d859e60c45822465f P a817c8ead3334f9e83210475da830972: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20251024 08:15:09.650274 2175 tablet.cc:2378] T e296ea373c1c488d859e60c45822465f P 65dce859a04548f3bf8cfecc41537241: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251024 08:15:09.661105 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 1784
I20251024 08:15:09.667747 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 1915
I20251024 08:15:09.673640 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 2046
I20251024 08:15:09.679787 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 1693
2025-10-24T08:15:09Z chronyd exiting
[ OK ] AdminCliTest.TestAuthzResetCacheNotAuthorized (875 ms)
[ RUN ] AdminCliTest.TestRebuildTables
I20251024 08:15:09.701823 32207 test_util.cc:276] Using random seed: 636056250
I20251024 08:15:09.702373 32207 ts_itest-base.cc:115] Starting cluster with:
I20251024 08:15:09.702426 32207 ts_itest-base.cc:116] --------------
I20251024 08:15:09.702438 32207 ts_itest-base.cc:117] 3 tablet servers
I20251024 08:15:09.702458 32207 ts_itest-base.cc:118] 3 replicas per TS
I20251024 08:15:09.702469 32207 ts_itest-base.cc:119] --------------
2025-10-24T08:15:09Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-24T08:15:09Z Disabled control of system clock
I20251024 08:15:09.711189 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:38491
--webserver_interface=127.31.115.254
--webserver_port=0
--builtin_ntp_servers=127.31.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:38491 with env {}
W20251024 08:15:09.797133 2223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:09.797360 2223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:09.797394 2223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:09.799141 2223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:09.799222 2223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:09.799247 2223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:09.799268 2223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:09.801206 2223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:38491
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:38491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.2223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:09.801556 2223 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:09.801818 2223 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:09.804594 2231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:09.804594 2228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:09.804594 2229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:09.804930 2223 server_base.cc:1047] running on GCE node
I20251024 08:15:09.805388 2223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:09.805675 2223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:09.806839 2223 hybrid_clock.cc:648] HybridClock initialized: now 1761293709806800 us; error 59 us; skew 500 ppm
I20251024 08:15:09.808408 2223 webserver.cc:492] Webserver started at http://127.31.115.254:45697/ using document root <none> and password file <none>
I20251024 08:15:09.808669 2223 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:09.808723 2223 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:09.808848 2223 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:09.809808 2223 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.810133 2223 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.811753 2223 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20251024 08:15:09.812732 2237 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.813020 2223 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:09.813126 2223 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.813207 2223 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:09.847128 2223 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:09.847465 2223 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:09.847594 2223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:09.851864 2223 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:38491
I20251024 08:15:09.851919 2289 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:38491 every 8 connection(s)
I20251024 08:15:09.852289 2223 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20251024 08:15:09.852955 2290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:09.855683 2290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Bootstrap starting.
I20251024 08:15:09.856344 2290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:09.856627 2290 log.cc:826] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:09.857363 2290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: No bootstrap required, opened a new log
I20251024 08:15:09.857405 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 2223
I20251024 08:15:09.857491 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance
I20251024 08:15:09.859046 2290 raft_consensus.cc:359] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:09.859196 2290 raft_consensus.cc:385] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:09.859230 2290 raft_consensus.cc:740] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8c0e3174ba4b0bbe6fcd5660edaec2, State: Initialized, Role: FOLLOWER
I20251024 08:15:09.859362 2290 consensus_queue.cc:260] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:09.859443 2290 raft_consensus.cc:399] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:09.859500 2290 raft_consensus.cc:493] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:09.859560 2290 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:09.860347 2290 raft_consensus.cc:515] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:09.860497 2290 leader_election.cc:304] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da8c0e3174ba4b0bbe6fcd5660edaec2; no voters:
I20251024 08:15:09.860671 2290 leader_election.cc:290] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:09.860747 2295 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:09.860939 2295 raft_consensus.cc:697] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 1 LEADER]: Becoming Leader. State: Replica: da8c0e3174ba4b0bbe6fcd5660edaec2, State: Running, Role: LEADER
I20251024 08:15:09.860961 2290 sys_catalog.cc:565] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:09.861040 2295 consensus_queue.cc:237] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:09.861353 2295 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da8c0e3174ba4b0bbe6fcd5660edaec2. Latest consensus state: current_term: 1 leader_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } } }
I20251024 08:15:09.861449 2295 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:09.861366 2296 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } } }
I20251024 08:15:09.861609 2296 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:09.861660 2300 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:09.862125 2300 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:09.864288 2300 catalog_manager.cc:1357] Generated new cluster ID: 33e4d088f012480aaf42f9095dbbb5e0
I20251024 08:15:09.864363 2300 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:09.879106 2300 catalog_manager.cc:1380] Generated new certificate authority record
I20251024 08:15:09.879765 2300 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:09.887817 2300 catalog_manager.cc:6022] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Generated new TSK 0
I20251024 08:15:09.888085 2300 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251024 08:15:09.893781 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.193:0
--local_ip_for_outbound_sockets=127.31.115.193
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:38491
--builtin_ntp_servers=127.31.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20251024 08:15:09.979714 2314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:09.979925 2314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:09.979961 2314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:09.981606 2314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:09.981699 2314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.193
I20251024 08:15:09.983556 2314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:38491
--never_fsync=true
--heap_profile_path=/tmp/kudu.2314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.193
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:09.983877 2314 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:09.984135 2314 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:09.986997 2322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:09.986989 2319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:09.987165 2314 server_base.cc:1047] running on GCE node
W20251024 08:15:09.986992 2320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:09.987543 2314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:09.987792 2314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:09.988997 2314 hybrid_clock.cc:648] HybridClock initialized: now 1761293709988968 us; error 45 us; skew 500 ppm
I20251024 08:15:09.991138 2314 webserver.cc:492] Webserver started at http://127.31.115.193:44521/ using document root <none> and password file <none>
I20251024 08:15:09.991423 2314 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:09.991482 2314 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:09.991616 2314 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:09.994385 2314 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "cffc63111053486b98e295df8d7ea55b"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.994886 2314 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "cffc63111053486b98e295df8d7ea55b"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.996376 2314 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:09.997300 2328 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:09.997538 2314 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:09.997627 2314 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "cffc63111053486b98e295df8d7ea55b"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:09.997720 2314 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:10.013191 2314 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:10.013502 2314 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:10.013638 2314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:10.013923 2314 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:10.014354 2314 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:10.014403 2314 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:10.014432 2314 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:10.014472 2314 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:10.022764 2314 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.193:44559
I20251024 08:15:10.022886 2441 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.193:44559 every 8 connection(s)
I20251024 08:15:10.023308 2314 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20251024 08:15:10.029500 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 2314
I20251024 08:15:10.029629 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20251024 08:15:10.030067 2442 heartbeater.cc:344] Connected to a master server at 127.31.115.254:38491
I20251024 08:15:10.030215 2442 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:10.030562 2442 heartbeater.cc:507] Master 127.31.115.254:38491 requested a full tablet report, sending...
I20251024 08:15:10.031157 2254 ts_manager.cc:194] Registered new tserver with Master: cffc63111053486b98e295df8d7ea55b (127.31.115.193:44559)
I20251024 08:15:10.031262 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.194:0
--local_ip_for_outbound_sockets=127.31.115.194
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:38491
--builtin_ntp_servers=127.31.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20251024 08:15:10.032212 2254 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.193:36433
W20251024 08:15:10.116056 2445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:10.116297 2445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:10.116339 2445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:10.118083 2445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:10.118167 2445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.194
I20251024 08:15:10.120036 2445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:38491
--never_fsync=true
--heap_profile_path=/tmp/kudu.2445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.194
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:10.120342 2445 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:10.120617 2445 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251024 08:15:10.123930 2445 server_base.cc:1047] running on GCE node
W20251024 08:15:10.123901 2450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:10.123894 2453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:10.123901 2451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:10.124543 2445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:10.124845 2445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:10.126017 2445 hybrid_clock.cc:648] HybridClock initialized: now 1761293710126001 us; error 49 us; skew 500 ppm
I20251024 08:15:10.127511 2445 webserver.cc:492] Webserver started at http://127.31.115.194:34127/ using document root <none> and password file <none>
I20251024 08:15:10.127743 2445 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:10.127797 2445 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:10.127940 2445 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:10.128891 2445 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "9928e83d100b4a4f82d2a1b53456fc35"
format_stamp: "Formatted at 2025-10-24 08:15:10 on dist-test-slave-c0j3"
I20251024 08:15:10.129266 2445 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "9928e83d100b4a4f82d2a1b53456fc35"
format_stamp: "Formatted at 2025-10-24 08:15:10 on dist-test-slave-c0j3"
I20251024 08:15:10.130649 2445 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20251024 08:15:10.131685 2459 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:10.131902 2445 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:10.131994 2445 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "9928e83d100b4a4f82d2a1b53456fc35"
format_stamp: "Formatted at 2025-10-24 08:15:10 on dist-test-slave-c0j3"
I20251024 08:15:10.132073 2445 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:10.143654 2445 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:10.143965 2445 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:10.144086 2445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:10.144325 2445 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:10.144652 2445 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:10.144686 2445 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:10.144727 2445 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:10.144760 2445 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:10.151008 2445 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.194:40709
I20251024 08:15:10.151072 2572 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.194:40709 every 8 connection(s)
I20251024 08:15:10.151422 2445 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20251024 08:15:10.156339 2573 heartbeater.cc:344] Connected to a master server at 127.31.115.254:38491
I20251024 08:15:10.156482 2573 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:10.156715 2573 heartbeater.cc:507] Master 127.31.115.254:38491 requested a full tablet report, sending...
I20251024 08:15:10.157114 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 2445
I20251024 08:15:10.157159 2254 ts_manager.cc:194] Registered new tserver with Master: 9928e83d100b4a4f82d2a1b53456fc35 (127.31.115.194:40709)
I20251024 08:15:10.157210 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20251024 08:15:10.157622 2254 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.194:57741
I20251024 08:15:10.158609 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.195:0
--local_ip_for_outbound_sockets=127.31.115.195
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:38491
--builtin_ntp_servers=127.31.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20251024 08:15:10.245088 2576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:10.245327 2576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:10.245352 2576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:10.247074 2576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:10.247165 2576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.195
I20251024 08:15:10.249002 2576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:38491
--never_fsync=true
--heap_profile_path=/tmp/kudu.2576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.195
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:10.249365 2576 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:10.249626 2576 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:10.252544 2584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:10.252652 2576 server_base.cc:1047] running on GCE node
W20251024 08:15:10.252529 2582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:10.252529 2581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:10.252931 2576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:10.253165 2576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:10.254372 2576 hybrid_clock.cc:648] HybridClock initialized: now 1761293710254352 us; error 37 us; skew 500 ppm
I20251024 08:15:10.256006 2576 webserver.cc:492] Webserver started at http://127.31.115.195:45711/ using document root <none> and password file <none>
I20251024 08:15:10.256258 2576 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:10.256314 2576 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:10.256465 2576 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:10.257393 2576 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "df8c042a3ff14ac380bb7f78b4c55171"
format_stamp: "Formatted at 2025-10-24 08:15:10 on dist-test-slave-c0j3"
I20251024 08:15:10.257730 2576 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "df8c042a3ff14ac380bb7f78b4c55171"
format_stamp: "Formatted at 2025-10-24 08:15:10 on dist-test-slave-c0j3"
I20251024 08:15:10.259163 2576 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20251024 08:15:10.260073 2590 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:10.260298 2576 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:10.260381 2576 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "df8c042a3ff14ac380bb7f78b4c55171"
format_stamp: "Formatted at 2025-10-24 08:15:10 on dist-test-slave-c0j3"
I20251024 08:15:10.260452 2576 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:10.271909 2576 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:10.272243 2576 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:10.272368 2576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:10.272595 2576 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:10.272926 2576 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:10.272960 2576 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:10.272998 2576 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:10.273029 2576 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:10.279206 2576 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.195:39423
I20251024 08:15:10.279286 2703 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.195:39423 every 8 connection(s)
I20251024 08:15:10.279628 2576 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20251024 08:15:10.283905 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 2576
I20251024 08:15:10.284020 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20251024 08:15:10.284837 2704 heartbeater.cc:344] Connected to a master server at 127.31.115.254:38491
I20251024 08:15:10.284962 2704 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:10.285229 2704 heartbeater.cc:507] Master 127.31.115.254:38491 requested a full tablet report, sending...
I20251024 08:15:10.285655 2253 ts_manager.cc:194] Registered new tserver with Master: df8c042a3ff14ac380bb7f78b4c55171 (127.31.115.195:39423)
I20251024 08:15:10.286077 2253 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.195:55565
I20251024 08:15:10.297065 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:10.303925 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:10.304008 32207 test_util.cc:276] Using random seed: 636658441
I20251024 08:15:10.310571 2253 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33710:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251024 08:15:10.318135 2376 tablet_service.cc:1505] Processing CreateTablet for tablet 67246a6574a64f2589345cd21c968dab (DEFAULT_TABLE table=TestTable [id=0778900937cb4d0a854c490de3fe53d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:10.318542 2376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67246a6574a64f2589345cd21c968dab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:10.320984 2719 tablet_bootstrap.cc:492] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Bootstrap starting.
I20251024 08:15:10.321704 2719 tablet_bootstrap.cc:654] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:10.322007 2719 log.cc:826] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:10.322844 2719 tablet_bootstrap.cc:492] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: No bootstrap required, opened a new log
I20251024 08:15:10.322921 2719 ts_tablet_manager.cc:1403] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:10.324097 2719 raft_consensus.cc:359] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } }
I20251024 08:15:10.324241 2719 raft_consensus.cc:385] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:10.324275 2719 raft_consensus.cc:740] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffc63111053486b98e295df8d7ea55b, State: Initialized, Role: FOLLOWER
I20251024 08:15:10.324379 2719 consensus_queue.cc:260] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } }
I20251024 08:15:10.324448 2719 raft_consensus.cc:399] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:10.324496 2719 raft_consensus.cc:493] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:10.324560 2719 raft_consensus.cc:3060] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:10.325163 2719 raft_consensus.cc:515] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } }
I20251024 08:15:10.325280 2719 leader_election.cc:304] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cffc63111053486b98e295df8d7ea55b; no voters:
I20251024 08:15:10.325498 2719 leader_election.cc:290] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:10.325619 2721 raft_consensus.cc:2804] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:10.325737 2719 ts_tablet_manager.cc:1434] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251024 08:15:10.325814 2721 raft_consensus.cc:697] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 1 LEADER]: Becoming Leader. State: Replica: cffc63111053486b98e295df8d7ea55b, State: Running, Role: LEADER
I20251024 08:15:10.325917 2442 heartbeater.cc:499] Master 127.31.115.254:38491 was elected leader, sending a full tablet report...
I20251024 08:15:10.325940 2721 consensus_queue.cc:237] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } }
I20251024 08:15:10.326946 2253 catalog_manager.cc:5649] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b reported cstate change: term changed from 0 to 1, leader changed from <none> to cffc63111053486b98e295df8d7ea55b (127.31.115.193). New cstate: current_term: 1 leader_uuid: "cffc63111053486b98e295df8d7ea55b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:10.352041 32207 test_util.cc:276] Using random seed: 636706454
I20251024 08:15:10.357123 2254 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33718:
name: "TestTable1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251024 08:15:10.362946 2638 tablet_service.cc:1505] Processing CreateTablet for tablet f50b7507c9c1415d947f2c28fee2828b (DEFAULT_TABLE table=TestTable1 [id=c306a00b365c4a38a0802725d4bcad4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:10.363302 2638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50b7507c9c1415d947f2c28fee2828b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:10.366060 2741 tablet_bootstrap.cc:492] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Bootstrap starting.
I20251024 08:15:10.366828 2741 tablet_bootstrap.cc:654] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:10.367184 2741 log.cc:826] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:10.368131 2741 tablet_bootstrap.cc:492] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: No bootstrap required, opened a new log
I20251024 08:15:10.368232 2741 ts_tablet_manager.cc:1403] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:10.369534 2741 raft_consensus.cc:359] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } }
I20251024 08:15:10.369700 2741 raft_consensus.cc:385] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:10.369721 2741 raft_consensus.cc:740] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df8c042a3ff14ac380bb7f78b4c55171, State: Initialized, Role: FOLLOWER
I20251024 08:15:10.369822 2741 consensus_queue.cc:260] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } }
I20251024 08:15:10.369890 2741 raft_consensus.cc:399] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:10.369922 2741 raft_consensus.cc:493] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:10.369987 2741 raft_consensus.cc:3060] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:10.370723 2741 raft_consensus.cc:515] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } }
I20251024 08:15:10.370852 2741 leader_election.cc:304] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df8c042a3ff14ac380bb7f78b4c55171; no voters:
I20251024 08:15:10.371042 2741 leader_election.cc:290] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:10.371115 2743 raft_consensus.cc:2804] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:10.371275 2741 ts_tablet_manager.cc:1434] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20251024 08:15:10.371305 2743 raft_consensus.cc:697] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 1 LEADER]: Becoming Leader. State: Replica: df8c042a3ff14ac380bb7f78b4c55171, State: Running, Role: LEADER
I20251024 08:15:10.371338 2704 heartbeater.cc:499] Master 127.31.115.254:38491 was elected leader, sending a full tablet report...
I20251024 08:15:10.371448 2743 consensus_queue.cc:237] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } }
I20251024 08:15:10.372212 2254 catalog_manager.cc:5649] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 reported cstate change: term changed from 0 to 1, leader changed from <none> to df8c042a3ff14ac380bb7f78b4c55171 (127.31.115.195). New cstate: current_term: 1 leader_uuid: "df8c042a3ff14ac380bb7f78b4c55171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:10.388427 32207 test_util.cc:276] Using random seed: 636742847
I20251024 08:15:10.394913 2247 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33732:
name: "TestTable2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251024 08:15:10.401830 2507 tablet_service.cc:1505] Processing CreateTablet for tablet 55fa483da9a74fc8849cfc952169890f (DEFAULT_TABLE table=TestTable2 [id=00d246e4c3424ea5a3b21ede80a5f873]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:10.402158 2507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55fa483da9a74fc8849cfc952169890f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:10.404582 2762 tablet_bootstrap.cc:492] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Bootstrap starting.
I20251024 08:15:10.405265 2762 tablet_bootstrap.cc:654] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:10.405547 2762 log.cc:826] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:10.407085 2762 tablet_bootstrap.cc:492] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: No bootstrap required, opened a new log
I20251024 08:15:10.407203 2762 ts_tablet_manager.cc:1403] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251024 08:15:10.408427 2762 raft_consensus.cc:359] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40709 } }
I20251024 08:15:10.408594 2762 raft_consensus.cc:385] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:10.408617 2762 raft_consensus.cc:740] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9928e83d100b4a4f82d2a1b53456fc35, State: Initialized, Role: FOLLOWER
I20251024 08:15:10.408721 2762 consensus_queue.cc:260] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40709 } }
I20251024 08:15:10.408800 2762 raft_consensus.cc:399] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:10.408826 2762 raft_consensus.cc:493] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:10.408862 2762 raft_consensus.cc:3060] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:10.409571 2762 raft_consensus.cc:515] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40709 } }
I20251024 08:15:10.409684 2762 leader_election.cc:304] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9928e83d100b4a4f82d2a1b53456fc35; no voters:
I20251024 08:15:10.409894 2762 leader_election.cc:290] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:10.410105 2762 ts_tablet_manager.cc:1434] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20251024 08:15:10.410254 2573 heartbeater.cc:499] Master 127.31.115.254:38491 was elected leader, sending a full tablet report...
I20251024 08:15:10.410264 2764 raft_consensus.cc:2804] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:10.410569 2764 raft_consensus.cc:697] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 1 LEADER]: Becoming Leader. State: Replica: 9928e83d100b4a4f82d2a1b53456fc35, State: Running, Role: LEADER
I20251024 08:15:10.410909 2764 consensus_queue.cc:237] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40709 } }
I20251024 08:15:10.412446 2247 catalog_manager.cc:5649] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9928e83d100b4a4f82d2a1b53456fc35 (127.31.115.194). New cstate: current_term: 1 leader_uuid: "9928e83d100b4a4f82d2a1b53456fc35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40709 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:10.428231 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 2223
I20251024 08:15:10.936062 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 2314
I20251024 08:15:10.941299 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 2445
I20251024 08:15:10.946230 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 2576
I20251024 08:15:10.951690 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:38491
--webserver_interface=127.31.115.254
--webserver_port=45697
--builtin_ntp_servers=127.31.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:38491 with env {}
W20251024 08:15:11.040568 2819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:11.040807 2819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:11.040841 2819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:11.042660 2819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:11.042734 2819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:11.042762 2819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:11.042783 2819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:11.044696 2819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:38491
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:38491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=45697
--never_fsync=true
--heap_profile_path=/tmp/kudu.2819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:11.045029 2819 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:11.045300 2819 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:11.047839 2824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:11.047858 2827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:11.047840 2825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:11.048158 2819 server_base.cc:1047] running on GCE node
I20251024 08:15:11.048363 2819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:11.048609 2819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:11.049784 2819 hybrid_clock.cc:648] HybridClock initialized: now 1761293711049762 us; error 37 us; skew 500 ppm
I20251024 08:15:11.051335 2819 webserver.cc:492] Webserver started at http://127.31.115.254:45697/ using document root <none> and password file <none>
I20251024 08:15:11.051591 2819 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:11.051652 2819 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:11.053308 2819 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:11.054281 2833 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:11.054713 2819 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:11.054824 2819 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:11.055140 2819 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:11.086689 2819 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:11.087018 2819 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:11.087147 2819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:11.091354 2819 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:38491
I20251024 08:15:11.091400 2885 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:38491 every 8 connection(s)
I20251024 08:15:11.091789 2819 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20251024 08:15:11.092978 2886 sys_catalog.cc:263] Verifying existing consensus state
I20251024 08:15:11.093626 2886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Bootstrap starting.
I20251024 08:15:11.095163 2886 log.cc:826] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:11.097620 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 2819
I20251024 08:15:11.097668 2886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=13 ignored=0} mutations{seen=10 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:11.097944 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.193:44559
--local_ip_for_outbound_sockets=127.31.115.193
--tserver_master_addrs=127.31.115.254:38491
--webserver_port=44521
--webserver_interface=127.31.115.193
--builtin_ntp_servers=127.31.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20251024 08:15:11.097992 2886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Bootstrap complete.
I20251024 08:15:11.100274 2886 raft_consensus.cc:359] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:11.100514 2886 raft_consensus.cc:740] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8c0e3174ba4b0bbe6fcd5660edaec2, State: Initialized, Role: FOLLOWER
I20251024 08:15:11.100653 2886 consensus_queue.cc:260] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:11.100724 2886 raft_consensus.cc:399] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:11.100766 2886 raft_consensus.cc:493] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:11.100817 2886 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 2 FOLLOWER]: Advancing to term 3
I20251024 08:15:11.101861 2886 raft_consensus.cc:515] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:11.101974 2886 leader_election.cc:304] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da8c0e3174ba4b0bbe6fcd5660edaec2; no voters:
I20251024 08:15:11.102138 2886 leader_election.cc:290] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20251024 08:15:11.102203 2890 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 3 FOLLOWER]: Leader election won for term 3
I20251024 08:15:11.102324 2890 raft_consensus.cc:697] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 3 LEADER]: Becoming Leader. State: Replica: da8c0e3174ba4b0bbe6fcd5660edaec2, State: Running, Role: LEADER
I20251024 08:15:11.102430 2886 sys_catalog.cc:565] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:11.102408 2890 consensus_queue.cc:237] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:11.102804 2890 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da8c0e3174ba4b0bbe6fcd5660edaec2. Latest consensus state: current_term: 3 leader_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } } }
I20251024 08:15:11.102883 2890 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:11.102849 2892 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } } }
I20251024 08:15:11.103418 2892 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:11.103948 2896 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:11.104678 2896 catalog_manager.cc:679] Loaded metadata for table TestTable2 [id=00d246e4c3424ea5a3b21ede80a5f873]
I20251024 08:15:11.104755 2896 catalog_manager.cc:679] Loaded metadata for table TestTable [id=95eb00caf0924d1c8f89ff4a61c1394a]
I20251024 08:15:11.104811 2896 catalog_manager.cc:679] Loaded metadata for table TestTable1 [id=ecee77a6eb90495bab69866e31e6194f]
I20251024 08:15:11.105034 2896 tablet_loader.cc:96] loaded metadata for tablet 55fa483da9a74fc8849cfc952169890f (table TestTable2 [id=00d246e4c3424ea5a3b21ede80a5f873])
I20251024 08:15:11.105094 2896 tablet_loader.cc:96] loaded metadata for tablet 67246a6574a64f2589345cd21c968dab (table TestTable [id=95eb00caf0924d1c8f89ff4a61c1394a])
I20251024 08:15:11.105149 2896 tablet_loader.cc:96] loaded metadata for tablet f50b7507c9c1415d947f2c28fee2828b (table TestTable1 [id=ecee77a6eb90495bab69866e31e6194f])
I20251024 08:15:11.105188 2896 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:11.105300 2896 catalog_manager.cc:1269] Loaded cluster ID: 33e4d088f012480aaf42f9095dbbb5e0
I20251024 08:15:11.105333 2896 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:11.106351 2896 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:11.106595 2896 catalog_manager.cc:6033] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Loaded TSK: 0
I20251024 08:15:11.106805 2896 catalog_manager.cc:1524] Initializing in-progress tserver states...
W20251024 08:15:11.185837 2889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:11.186064 2889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:11.186086 2889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:11.187884 2889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:11.187985 2889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.193
I20251024 08:15:11.189823 2889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.193:44559
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.115.193
--webserver_port=44521
--tserver_master_addrs=127.31.115.254:38491
--never_fsync=true
--heap_profile_path=/tmp/kudu.2889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.193
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:11.190125 2889 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:11.190387 2889 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:11.193260 2913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:11.193367 2915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:11.193259 2912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:11.193639 2889 server_base.cc:1047] running on GCE node
I20251024 08:15:11.193817 2889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:11.194059 2889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:11.195219 2889 hybrid_clock.cc:648] HybridClock initialized: now 1761293711195205 us; error 37 us; skew 500 ppm
I20251024 08:15:11.196580 2889 webserver.cc:492] Webserver started at http://127.31.115.193:44521/ using document root <none> and password file <none>
I20251024 08:15:11.196815 2889 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:11.196889 2889 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:11.198309 2889 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:11.199313 2921 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:11.199568 2889 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:11.199651 2889 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "cffc63111053486b98e295df8d7ea55b"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:11.199954 2889 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:11.218657 2889 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:11.218988 2889 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:11.219121 2889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:11.219364 2889 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:11.219875 2928 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20251024 08:15:11.220774 2889 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251024 08:15:11.220832 2889 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:11.220861 2889 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251024 08:15:11.221453 2889 ts_tablet_manager.cc:616] Registered 1 tablets
I20251024 08:15:11.221499 2889 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:11.221556 2928 tablet_bootstrap.cc:492] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Bootstrap starting.
I20251024 08:15:11.230544 2889 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.193:44559
I20251024 08:15:11.231138 2889 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20251024 08:15:11.234118 2928 log.cc:826] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:11.234110 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 2889
I20251024 08:15:11.234467 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.194:40709
--local_ip_for_outbound_sockets=127.31.115.194
--tserver_master_addrs=127.31.115.254:38491
--webserver_port=34127
--webserver_interface=127.31.115.194
--builtin_ntp_servers=127.31.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20251024 08:15:11.234930 3035 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.193:44559 every 8 connection(s)
I20251024 08:15:11.243692 2928 tablet_bootstrap.cc:492] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=1400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:11.244436 2928 tablet_bootstrap.cc:492] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Bootstrap complete.
I20251024 08:15:11.245707 2928 ts_tablet_manager.cc:1403] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.014s
I20251024 08:15:11.246265 3036 heartbeater.cc:344] Connected to a master server at 127.31.115.254:38491
I20251024 08:15:11.246381 3036 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:11.246707 3036 heartbeater.cc:507] Master 127.31.115.254:38491 requested a full tablet report, sending...
I20251024 08:15:11.247249 2928 raft_consensus.cc:359] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } }
I20251024 08:15:11.247581 2928 raft_consensus.cc:740] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffc63111053486b98e295df8d7ea55b, State: Initialized, Role: FOLLOWER
I20251024 08:15:11.247622 2850 ts_manager.cc:194] Registered new tserver with Master: cffc63111053486b98e295df8d7ea55b (127.31.115.193:44559)
I20251024 08:15:11.247715 2928 consensus_queue.cc:260] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } }
I20251024 08:15:11.247772 2928 raft_consensus.cc:399] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:11.247809 2928 raft_consensus.cc:493] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:11.247864 2928 raft_consensus.cc:3060] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:11.248742 2850 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.193:60729
I20251024 08:15:11.249380 2928 raft_consensus.cc:515] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } }
I20251024 08:15:11.249541 2928 leader_election.cc:304] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cffc63111053486b98e295df8d7ea55b; no voters:
I20251024 08:15:11.249727 2928 leader_election.cc:290] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [CANDIDATE]: Term 2 election: Requested vote from peers
I20251024 08:15:11.249965 2928 ts_tablet_manager.cc:1434] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20251024 08:15:11.250263 3043 raft_consensus.cc:2804] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 2 FOLLOWER]: Leader election won for term 2
I20251024 08:15:11.250387 3036 heartbeater.cc:499] Master 127.31.115.254:38491 was elected leader, sending a full tablet report...
I20251024 08:15:11.250386 3043 raft_consensus.cc:697] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 2 LEADER]: Becoming Leader. State: Replica: cffc63111053486b98e295df8d7ea55b, State: Running, Role: LEADER
I20251024 08:15:11.250612 3043 consensus_queue.cc:237] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } }
I20251024 08:15:11.251371 2850 catalog_manager.cc:5649] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b reported cstate change: term changed from 0 to 2, leader changed from <none> to cffc63111053486b98e295df8d7ea55b (127.31.115.193), VOTER cffc63111053486b98e295df8d7ea55b (127.31.115.193) added. New cstate: current_term: 2 leader_uuid: "cffc63111053486b98e295df8d7ea55b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } health_report { overall_health: HEALTHY } } }
W20251024 08:15:11.254685 2850 catalog_manager.cc:5327] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 67246a6574a64f2589345cd21c968dab with cas_config_opid_index -1: no extra replica candidate found for tablet 67246a6574a64f2589345cd21c968dab (table TestTable [id=95eb00caf0924d1c8f89ff4a61c1394a]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (1) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
W20251024 08:15:11.327108 3041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:11.327332 3041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:11.327354 3041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:11.328943 3041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:11.329018 3041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.194
I20251024 08:15:11.330796 3041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.194:40709
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.115.194
--webserver_port=34127
--tserver_master_addrs=127.31.115.254:38491
--never_fsync=true
--heap_profile_path=/tmp/kudu.3041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.194
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:11.331068 3041 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:11.331305 3041 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:11.334474 3055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:11.334609 3041 server_base.cc:1047] running on GCE node
W20251024 08:15:11.334463 3053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:11.334474 3052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:11.335117 3041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:11.335402 3041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:11.336601 3041 hybrid_clock.cc:648] HybridClock initialized: now 1761293711336579 us; error 59 us; skew 500 ppm
I20251024 08:15:11.338148 3041 webserver.cc:492] Webserver started at http://127.31.115.194:34127/ using document root <none> and password file <none>
I20251024 08:15:11.338379 3041 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:11.338459 3041 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:11.339879 3041 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:11.340667 3061 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:11.340850 3041 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:11.340934 3041 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "9928e83d100b4a4f82d2a1b53456fc35"
format_stamp: "Formatted at 2025-10-24 08:15:10 on dist-test-slave-c0j3"
I20251024 08:15:11.341233 3041 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:11.355350 3041 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:11.355674 3041 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:11.355801 3041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:11.356035 3041 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:11.356480 3068 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20251024 08:15:11.357473 3041 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251024 08:15:11.357538 3041 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:11.357594 3041 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251024 08:15:11.358246 3041 ts_tablet_manager.cc:616] Registered 1 tablets
I20251024 08:15:11.358291 3041 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:11.358340 3068 tablet_bootstrap.cc:492] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Bootstrap starting.
I20251024 08:15:11.365311 3041 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.194:40709
I20251024 08:15:11.365417 3175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.194:40709 every 8 connection(s)
I20251024 08:15:11.365752 3041 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20251024 08:15:11.369549 3068 log.cc:826] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:11.371626 3176 heartbeater.cc:344] Connected to a master server at 127.31.115.254:38491
I20251024 08:15:11.371754 3176 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:11.371973 3176 heartbeater.cc:507] Master 127.31.115.254:38491 requested a full tablet report, sending...
I20251024 08:15:11.372525 2850 ts_manager.cc:194] Registered new tserver with Master: 9928e83d100b4a4f82d2a1b53456fc35 (127.31.115.194:40709)
I20251024 08:15:11.373081 2850 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.194:33303
I20251024 08:15:11.373208 3068 tablet_bootstrap.cc:492] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=300 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:11.373519 3068 tablet_bootstrap.cc:492] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Bootstrap complete.
I20251024 08:15:11.373740 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 3041
I20251024 08:15:11.373997 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.195:39423
--local_ip_for_outbound_sockets=127.31.115.195
--tserver_master_addrs=127.31.115.254:38491
--webserver_port=45711
--webserver_interface=127.31.115.195
--builtin_ntp_servers=127.31.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20251024 08:15:11.374554 3068 ts_tablet_manager.cc:1403] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.004s
I20251024 08:15:11.375488 3068 raft_consensus.cc:359] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40709 } }
I20251024 08:15:11.375705 3068 raft_consensus.cc:740] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9928e83d100b4a4f82d2a1b53456fc35, State: Initialized, Role: FOLLOWER
I20251024 08:15:11.375835 3068 consensus_queue.cc:260] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40709 } }
I20251024 08:15:11.375897 3068 raft_consensus.cc:399] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:11.375933 3068 raft_consensus.cc:493] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:11.375991 3068 raft_consensus.cc:3060] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:11.377033 3068 raft_consensus.cc:515] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40709 } }
I20251024 08:15:11.377177 3068 leader_election.cc:304] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9928e83d100b4a4f82d2a1b53456fc35; no voters:
I20251024 08:15:11.377364 3068 leader_election.cc:290] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [CANDIDATE]: Term 2 election: Requested vote from peers
I20251024 08:15:11.377429 3181 raft_consensus.cc:2804] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 2 FOLLOWER]: Leader election won for term 2
I20251024 08:15:11.377548 3181 raft_consensus.cc:697] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [term 2 LEADER]: Becoming Leader. State: Replica: 9928e83d100b4a4f82d2a1b53456fc35, State: Running, Role: LEADER
I20251024 08:15:11.377601 3068 ts_tablet_manager.cc:1434] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251024 08:15:11.377715 3176 heartbeater.cc:499] Master 127.31.115.254:38491 was elected leader, sending a full tablet report...
I20251024 08:15:11.377664 3181 consensus_queue.cc:237] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40709 } }
I20251024 08:15:11.378597 2850 catalog_manager.cc:5649] T 55fa483da9a74fc8849cfc952169890f P 9928e83d100b4a4f82d2a1b53456fc35 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "9928e83d100b4a4f82d2a1b53456fc35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40709 } health_report { overall_health: HEALTHY } } }
W20251024 08:15:11.460495 3180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:11.460718 3180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:11.460754 3180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:11.462358 3180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:11.462448 3180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.195
I20251024 08:15:11.464439 3180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.195:39423
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.115.195
--webserver_port=45711
--tserver_master_addrs=127.31.115.254:38491
--never_fsync=true
--heap_profile_path=/tmp/kudu.3180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.195
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:11.464787 3180 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:11.465058 3180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251024 08:15:11.466534 2990 consensus_queue.cc:237] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 29, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } }
I20251024 08:15:11.467103 3044 raft_consensus.cc:2955] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index -1 to 31, NON_VOTER 9928e83d100b4a4f82d2a1b53456fc35 (127.31.115.194) added. New config: { opid_index: 31 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } } }
W20251024 08:15:11.468253 3192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:11.468248 3193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:11.468825 2834 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 67246a6574a64f2589345cd21c968dab with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 4)
W20251024 08:15:11.468927 3195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:11.468940 2850 catalog_manager.cc:5649] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b reported cstate change: config changed from index -1 to 31, NON_VOTER 9928e83d100b4a4f82d2a1b53456fc35 (127.31.115.194) added. New cstate: current_term: 2 leader_uuid: "cffc63111053486b98e295df8d7ea55b" committed_config { opid_index: 31 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251024 08:15:11.469558 3180 server_base.cc:1047] running on GCE node
I20251024 08:15:11.469774 3180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:11.470037 3180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:11.471196 3180 hybrid_clock.cc:648] HybridClock initialized: now 1761293711471177 us; error 39 us; skew 500 ppm
W20251024 08:15:11.472160 2924 consensus_peers.cc:597] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b -> Peer 9928e83d100b4a4f82d2a1b53456fc35 (127.31.115.194:40709): Couldn't send request to peer 9928e83d100b4a4f82d2a1b53456fc35. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 67246a6574a64f2589345cd21c968dab. This is attempt 1: this message will repeat every 5th retry.
I20251024 08:15:11.472651 3180 webserver.cc:492] Webserver started at http://127.31.115.195:45711/ using document root <none> and password file <none>
I20251024 08:15:11.472865 3180 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:11.472934 3180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:11.474290 3180 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:11.475482 3202 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:11.475852 3180 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:11.475955 3180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "df8c042a3ff14ac380bb7f78b4c55171"
format_stamp: "Formatted at 2025-10-24 08:15:10 on dist-test-slave-c0j3"
I20251024 08:15:11.476274 3180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:11.501585 3180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:11.501896 3180 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:11.502017 3180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:11.502242 3180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:11.502776 3209 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20251024 08:15:11.503790 3180 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251024 08:15:11.503847 3180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:11.503897 3180 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251024 08:15:11.504565 3180 ts_tablet_manager.cc:616] Registered 1 tablets
I20251024 08:15:11.504601 3180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:11.504654 3209 tablet_bootstrap.cc:492] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Bootstrap starting.
I20251024 08:15:11.511986 3180 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.195:39423
I20251024 08:15:11.512082 3316 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.195:39423 every 8 connection(s)
I20251024 08:15:11.512440 3180 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20251024 08:15:11.515291 3209 log.cc:826] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:11.517787 3317 heartbeater.cc:344] Connected to a master server at 127.31.115.254:38491
I20251024 08:15:11.517930 3317 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:11.518179 3317 heartbeater.cc:507] Master 127.31.115.254:38491 requested a full tablet report, sending...
I20251024 08:15:11.518867 2850 ts_manager.cc:194] Registered new tserver with Master: df8c042a3ff14ac380bb7f78b4c55171 (127.31.115.195:39423)
I20251024 08:15:11.519448 2850 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.195:32769
I20251024 08:15:11.519476 3209 tablet_bootstrap.cc:492] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:11.519748 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 3180
I20251024 08:15:11.519834 3209 tablet_bootstrap.cc:492] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Bootstrap complete.
I20251024 08:15:11.520836 3209 ts_tablet_manager.cc:1403] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.008s
I20251024 08:15:11.521806 3209 raft_consensus.cc:359] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } }
I20251024 08:15:11.522097 3209 raft_consensus.cc:740] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: df8c042a3ff14ac380bb7f78b4c55171, State: Initialized, Role: FOLLOWER
I20251024 08:15:11.522230 3209 consensus_queue.cc:260] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } }
I20251024 08:15:11.522300 3209 raft_consensus.cc:399] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:11.522347 3209 raft_consensus.cc:493] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:11.522292 2990 consensus_queue.cc:237] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 29, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } } peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: NON_VOTER last_known_addr { host: "127.31.115.195" port: 39423 } attrs { promote: true } }
I20251024 08:15:11.522397 3209 raft_consensus.cc:3060] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:11.523062 3043 raft_consensus.cc:2955] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, NON_VOTER df8c042a3ff14ac380bb7f78b4c55171 (127.31.115.195) added. New config: { opid_index: 32 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } } peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: NON_VOTER last_known_addr { host: "127.31.115.195" port: 39423 } attrs { promote: true } } }
I20251024 08:15:11.523906 3209 raft_consensus.cc:515] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } }
I20251024 08:15:11.524031 3209 leader_election.cc:304] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df8c042a3ff14ac380bb7f78b4c55171; no voters:
I20251024 08:15:11.524209 3209 leader_election.cc:290] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [CANDIDATE]: Term 2 election: Requested vote from peers
I20251024 08:15:11.524281 2850 catalog_manager.cc:5649] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b reported cstate change: config changed from index 31 to 32, NON_VOTER df8c042a3ff14ac380bb7f78b4c55171 (127.31.115.195) added. New cstate: current_term: 2 leader_uuid: "cffc63111053486b98e295df8d7ea55b" committed_config { opid_index: 32 OBSOLETE_local: true peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 44559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: NON_VOTER last_known_addr { host: "127.31.115.195" port: 39423 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251024 08:15:11.524441 3209 ts_tablet_manager.cc:1434] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Time spent starting tablet: real 0.004s user 0.005s sys 0.001s
I20251024 08:15:11.524490 2834 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 67246a6574a64f2589345cd21c968dab with cas_config_opid_index 31: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 2)
I20251024 08:15:11.524477 3324 raft_consensus.cc:2804] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 2 FOLLOWER]: Leader election won for term 2
W20251024 08:15:11.524622 2924 consensus_peers.cc:597] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b -> Peer 9928e83d100b4a4f82d2a1b53456fc35 (127.31.115.194:40709): Couldn't send request to peer 9928e83d100b4a4f82d2a1b53456fc35. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 67246a6574a64f2589345cd21c968dab. This is attempt 1: this message will repeat every 5th retry.
I20251024 08:15:11.524637 3317 heartbeater.cc:499] Master 127.31.115.254:38491 was elected leader, sending a full tablet report...
I20251024 08:15:11.524816 3324 raft_consensus.cc:697] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 2 LEADER]: Becoming Leader. State: Replica: df8c042a3ff14ac380bb7f78b4c55171, State: Running, Role: LEADER
I20251024 08:15:11.524976 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:11.524988 3324 consensus_queue.cc:237] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } }
I20251024 08:15:11.525733 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:11.525921 2850 catalog_manager.cc:5649] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 reported cstate change: term changed from 0 to 2, leader changed from <none> to df8c042a3ff14ac380bb7f78b4c55171 (127.31.115.195), VOTER df8c042a3ff14ac380bb7f78b4c55171 (127.31.115.195) added. New cstate: current_term: 2 leader_uuid: "df8c042a3ff14ac380bb7f78b4c55171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:11.527380 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver cffc63111053486b98e295df8d7ea55b to finish bootstrapping
W20251024 08:15:11.528707 2923 consensus_peers.cc:597] T 67246a6574a64f2589345cd21c968dab P cffc63111053486b98e295df8d7ea55b -> Peer df8c042a3ff14ac380bb7f78b4c55171 (127.31.115.195:39423): Couldn't send request to peer df8c042a3ff14ac380bb7f78b4c55171. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 67246a6574a64f2589345cd21c968dab. This is attempt 1: this message will repeat every 5th retry.
I20251024 08:15:11.530794 3271 consensus_queue.cc:237] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } }
I20251024 08:15:11.531325 3324 raft_consensus.cc:2955] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index -1 to 11, NON_VOTER 9928e83d100b4a4f82d2a1b53456fc35 (127.31.115.194) added. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } } }
I20251024 08:15:11.531684 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 9928e83d100b4a4f82d2a1b53456fc35 to finish bootstrapping
I20251024 08:15:11.532282 2835 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f50b7507c9c1415d947f2c28fee2828b with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251024 08:15:11.532474 2850 catalog_manager.cc:5649] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 reported cstate change: config changed from index -1 to 11, NON_VOTER 9928e83d100b4a4f82d2a1b53456fc35 (127.31.115.194) added. New cstate: current_term: 2 leader_uuid: "df8c042a3ff14ac380bb7f78b4c55171" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251024 08:15:11.533636 3271 consensus_queue.cc:237] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } } peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: NON_VOTER last_known_addr { host: "127.31.115.193" port: 44559 } attrs { promote: true } }
I20251024 08:15:11.533984 3324 raft_consensus.cc:2955] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 [term 2 LEADER]: Committing config change with OpId 2.12: config changed from index 11 to 12, NON_VOTER cffc63111053486b98e295df8d7ea55b (127.31.115.193) added. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } } peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: NON_VOTER last_known_addr { host: "127.31.115.193" port: 44559 } attrs { promote: true } } }
I20251024 08:15:11.535207 2835 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f50b7507c9c1415d947f2c28fee2828b with cas_config_opid_index 11: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251024 08:15:11.535318 2850 catalog_manager.cc:5649] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 reported cstate change: config changed from index 11 to 12, NON_VOTER cffc63111053486b98e295df8d7ea55b (127.31.115.193) added. New cstate: current_term: 2 leader_uuid: "df8c042a3ff14ac380bb7f78b4c55171" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: NON_VOTER last_known_addr { host: "127.31.115.193" port: 44559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251024 08:15:11.536619 3205 consensus_peers.cc:597] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 -> Peer 9928e83d100b4a4f82d2a1b53456fc35 (127.31.115.194:40709): Couldn't send request to peer 9928e83d100b4a4f82d2a1b53456fc35. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f50b7507c9c1415d947f2c28fee2828b. This is attempt 1: this message will repeat every 5th retry.
I20251024 08:15:11.537125 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver df8c042a3ff14ac380bb7f78b4c55171 to finish bootstrapping
W20251024 08:15:11.538256 3203 consensus_peers.cc:597] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171 -> Peer cffc63111053486b98e295df8d7ea55b (127.31.115.193:44559): Couldn't send request to peer cffc63111053486b98e295df8d7ea55b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f50b7507c9c1415d947f2c28fee2828b. This is attempt 1: this message will repeat every 5th retry.
I20251024 08:15:11.571548 3251 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251024 08:15:11.571993 2970 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251024 08:15:11.572350 3110 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
da8c0e3174ba4b0bbe6fcd5660edaec2 | 127.31.115.254:38491 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.31.115.212:46415 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
9928e83d100b4a4f82d2a1b53456fc35 | 127.31.115.194:40709 | HEALTHY | <none> | 1 | 0
cffc63111053486b98e295df8d7ea55b | 127.31.115.193:44559 | HEALTHY | <none> | 1 | 0
df8c042a3ff14ac380bb7f78b4c55171 | 127.31.115.195:39423 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.31.115.193 | experimental | 127.31.115.193:44559
local_ip_for_outbound_sockets | 127.31.115.194 | experimental | 127.31.115.194:40709
local_ip_for_outbound_sockets | 127.31.115.195 | experimental | 127.31.115.195:39423
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.31.115.193:44559
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.31.115.194:40709
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.31.115.195:39423
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.31.115.212:46415 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
TestTable1 | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
TestTable2 | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 3
Maximum | 3
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 3
Tablets | 3
Replicas | 7
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20251024 08:15:11.583704 32207 log_verifier.cc:126] Checking tablet 55fa483da9a74fc8849cfc952169890f
I20251024 08:15:11.591553 32207 log_verifier.cc:177] Verified matching terms for 8 ops in tablet 55fa483da9a74fc8849cfc952169890f
I20251024 08:15:11.591631 32207 log_verifier.cc:126] Checking tablet 67246a6574a64f2589345cd21c968dab
I20251024 08:15:11.598889 32207 log_verifier.cc:177] Verified matching terms for 32 ops in tablet 67246a6574a64f2589345cd21c968dab
I20251024 08:15:11.598959 32207 log_verifier.cc:126] Checking tablet f50b7507c9c1415d947f2c28fee2828b
I20251024 08:15:11.605427 32207 log_verifier.cc:177] Verified matching terms for 12 ops in tablet f50b7507c9c1415d947f2c28fee2828b
I20251024 08:15:11.605559 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 2819
I20251024 08:15:11.611675 32207 minidump.cc:252] Setting minidump size limit to 20M
I20251024 08:15:11.611981 32207 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:11.612123 32207 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:11.614904 3357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:11.614902 3356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:11.614939 3359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:11.614939 32207 server_base.cc:1047] running on GCE node
I20251024 08:15:11.615571 32207 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251024 08:15:11.615633 32207 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251024 08:15:11.615664 32207 hybrid_clock.cc:648] HybridClock initialized: now 1761293711615664 us; error 0 us; skew 500 ppm
I20251024 08:15:11.616580 32207 webserver.cc:492] Webserver started at http://0.0.0.0:43111/ using document root <none> and password file <none>
I20251024 08:15:11.616791 32207 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:11.616832 32207 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:11.617866 32207 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:11.618671 3364 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:11.618858 32207 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:11.618952 32207 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2"
format_stamp: "Formatted at 2025-10-24 08:15:09 on dist-test-slave-c0j3"
I20251024 08:15:11.619189 32207 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:11.624841 32207 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:11.625149 32207 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:11.625274 32207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:11.626258 32207 sys_catalog.cc:263] Verifying existing consensus state
W20251024 08:15:11.626850 32207 sys_catalog.cc:243] For a single master config, on-disk Raft master: 127.31.115.254:38491 exists but no master address supplied!
I20251024 08:15:11.627524 32207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Bootstrap starting.
I20251024 08:15:11.636072 32207 log.cc:826] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:11.638865 32207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Bootstrap replayed 1/1 log segments. Stats: ops{read=26 overwritten=0 applied=26 ignored=0} inserts{seen=13 ignored=0} mutations{seen=17 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:11.639197 32207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2: Bootstrap complete.
I20251024 08:15:11.641034 32207 raft_consensus.cc:359] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:11.641171 32207 raft_consensus.cc:740] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8c0e3174ba4b0bbe6fcd5660edaec2, State: Initialized, Role: FOLLOWER
I20251024 08:15:11.641283 32207 consensus_queue.cc:260] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 26, Last appended: 3.26, Last appended by leader: 26, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:11.641341 32207 raft_consensus.cc:399] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:11.641396 32207 raft_consensus.cc:493] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:11.641450 32207 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 3 FOLLOWER]: Advancing to term 4
I20251024 08:15:11.642392 32207 raft_consensus.cc:515] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:11.642550 32207 leader_election.cc:304] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da8c0e3174ba4b0bbe6fcd5660edaec2; no voters:
I20251024 08:15:11.642755 32207 leader_election.cc:290] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [CANDIDATE]: Term 4 election: Requested vote from peers
I20251024 08:15:11.642843 3371 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 4 FOLLOWER]: Leader election won for term 4
I20251024 08:15:11.643038 3371 raft_consensus.cc:697] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 4 LEADER]: Becoming Leader. State: Replica: da8c0e3174ba4b0bbe6fcd5660edaec2, State: Running, Role: LEADER
I20251024 08:15:11.643144 3371 consensus_queue.cc:237] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 3.26, Last appended by leader: 26, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } }
I20251024 08:15:11.643450 3372 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 leader_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } } }
I20251024 08:15:11.643522 3373 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da8c0e3174ba4b0bbe6fcd5660edaec2. Latest consensus state: current_term: 4 leader_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8c0e3174ba4b0bbe6fcd5660edaec2" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 38491 } } }
I20251024 08:15:11.643546 3372 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:11.643597 3373 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:11.645397 32207 tablet_replica.cc:333] stopping tablet replica
I20251024 08:15:11.645509 32207 raft_consensus.cc:2243] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 4 LEADER]: Raft consensus shutting down.
I20251024 08:15:11.645573 32207 raft_consensus.cc:2272] T 00000000000000000000000000000000 P da8c0e3174ba4b0bbe6fcd5660edaec2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251024 08:15:11.645932 32207 master.cc:561] Master@0.0.0.0:7051 shutting down...
W20251024 08:15:11.646064 32207 acceptor_pool.cc:194] Could not shut down acceptor socket on 0.0.0.0:7051: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20251024 08:15:11.657079 32207 master.cc:583] Master@0.0.0.0:7051 shutdown complete.
I20251024 08:15:12.011682 3421 ts_tablet_manager.cc:933] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35: Initiating tablet copy from peer df8c042a3ff14ac380bb7f78b4c55171 (127.31.115.195:39423)
I20251024 08:15:12.011914 3421 tablet_copy_client.cc:323] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35: tablet copy: Beginning tablet copy session from remote peer at address 127.31.115.195:39423
I20251024 08:15:12.015327 3291 tablet_copy_service.cc:140] P df8c042a3ff14ac380bb7f78b4c55171: Received BeginTabletCopySession request for tablet f50b7507c9c1415d947f2c28fee2828b from peer 9928e83d100b4a4f82d2a1b53456fc35 ({username='slave'} at 127.31.115.194:38307)
I20251024 08:15:12.015434 3291 tablet_copy_service.cc:161] P df8c042a3ff14ac380bb7f78b4c55171: Beginning new tablet copy session on tablet f50b7507c9c1415d947f2c28fee2828b from peer 9928e83d100b4a4f82d2a1b53456fc35 at {username='slave'} at 127.31.115.194:38307: session id = 9928e83d100b4a4f82d2a1b53456fc35-f50b7507c9c1415d947f2c28fee2828b
I20251024 08:15:12.015936 3291 tablet_copy_source_session.cc:215] T f50b7507c9c1415d947f2c28fee2828b P df8c042a3ff14ac380bb7f78b4c55171: Tablet Copy: opened 0 blocks and 1 log segments
I20251024 08:15:12.016669 3421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50b7507c9c1415d947f2c28fee2828b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:12.018162 3421 tablet_copy_client.cc:806] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35: tablet copy: Starting download of 0 data blocks...
I20251024 08:15:12.018316 3421 tablet_copy_client.cc:670] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35: tablet copy: Starting download of 1 WAL segments...
I20251024 08:15:12.019312 3421 tablet_copy_client.cc:538] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251024 08:15:12.020054 3421 tablet_bootstrap.cc:492] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35: Bootstrap starting.
I20251024 08:15:12.022887 3421 tablet_bootstrap.cc:492] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:12.023115 3421 tablet_bootstrap.cc:492] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35: Bootstrap complete.
I20251024 08:15:12.023243 3421 ts_tablet_manager.cc:1403] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251024 08:15:12.023406 3421 raft_consensus.cc:359] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } } peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: NON_VOTER last_known_addr { host: "127.31.115.193" port: 44559 } attrs { promote: true } }
I20251024 08:15:12.023483 3421 raft_consensus.cc:740] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 9928e83d100b4a4f82d2a1b53456fc35, State: Initialized, Role: LEARNER
I20251024 08:15:12.023510 3421 consensus_queue.cc:260] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 2.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "df8c042a3ff14ac380bb7f78b4c55171" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 39423 } } peers { permanent_uuid: "9928e83d100b4a4f82d2a1b53456fc35" member_type: NON_VOTER last_known_addr { host: "127.31.115.194" port: 40709 } attrs { promote: true } } peers { permanent_uuid: "cffc63111053486b98e295df8d7ea55b" member_type: NON_VOTER last_known_addr { host: "127.31.115.193" port: 44559 } attrs { promote: true } }
I20251024 08:15:12.023739 3421 ts_tablet_manager.cc:1434] T f50b7507c9c1415d947f2c28fee2828b P 9928e83d100b4a4f82d2a1b53456fc35: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251024 08:15:12.024161 3291 tablet_copy_service.cc:342] P df8c042a3ff14ac380bb7f78b4c55171: Request end of tablet copy session 9928e83d100b4a4f82d2a1b53456fc35-f50b7507c9c1415d947f2c28fee2828b received from {username='slave'} at 127.31.115.194:38307
I20251024 08:15:12.024242 3291 tablet_copy_service.cc:434] P df8c042a3ff14ac380bb7f78b4c55171: ending tablet copy session 9928e83d100b4a4f82d2a1b53456fc35-f50b7507c9c1415d947f2c28fee2828b on tablet f50b7507c9c1415d947f2c28fee2828b with peer 9928e83d100b4a4f82d2a1b53456fc35
W20251024 08:15:12.024245 3176 heartbeater.cc:646] Failed to heartbeat to 127.31.115.254:38491 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.115.254:38491: connect: Connection refused (error 111)
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tools/kudu-admin-test.cc:3916: Failure
Value of: stdout2
Expected: has substring "Rebuilt from 3 replicas, of which 0 had errors"
Actual: "Rebuild Report\nTablet Servers\n address | status\n----------------------+--------\n 127.31.115.193:44559 | OK\n 127.31.115.194:40709 | OK\n 127.31.115.195:39423 | OK\nRebuilt from 3 tablet servers, of which 0 had errors\n\nReplicas\n table | tablet | tablet server | status\n-----------+----------------------------------+----------------------+--------\n TestTable | 67246a6574a64f2589345cd21c968dab | 127.31.115.193:44559 | OK\nRebuilt from 1 replicas, of which 0 had errors\n"
I20251024 08:15:12.052853 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 2889
I20251024 08:15:12.059149 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 3041
I20251024 08:15:12.065332 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 3180
2025-10-24T08:15:12Z chronyd exiting
I20251024 08:15:12.081502 32207 test_util.cc:183] -----------------------------------------------
I20251024 08:15:12.081606 32207 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1761293700605927-32207-0
[ FAILED ] AdminCliTest.TestRebuildTables (2380 ms)
[----------] 5 tests from AdminCliTest (11469 ms total)
[----------] 1 test from EnableKudu1097AndDownTS/MoveTabletParamTest
[ RUN ] EnableKudu1097AndDownTS/MoveTabletParamTest.Test/4
I20251024 08:15:12.082118 32207 test_util.cc:276] Using random seed: 638436549
I20251024 08:15:12.083048 32207 ts_itest-base.cc:115] Starting cluster with:
I20251024 08:15:12.083092 32207 ts_itest-base.cc:116] --------------
I20251024 08:15:12.083113 32207 ts_itest-base.cc:117] 5 tablet servers
I20251024 08:15:12.083129 32207 ts_itest-base.cc:118] 3 replicas per TS
I20251024 08:15:12.083148 32207 ts_itest-base.cc:119] --------------
2025-10-24T08:15:12Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-24T08:15:12Z Disabled control of system clock
I20251024 08:15:12.092028 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:45843
--webserver_interface=127.31.115.254
--webserver_port=0
--builtin_ntp_servers=127.31.115.212:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:45843
--raft_prepare_replacement_before_eviction=true with env {}
W20251024 08:15:12.176534 3434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:12.176766 3434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:12.176800 3434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:12.178463 3434 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20251024 08:15:12.178576 3434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:12.178606 3434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:12.178630 3434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:12.178648 3434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:12.180488 3434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:45843
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:45843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.3434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:12.180816 3434 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:12.181082 3434 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:12.183797 3439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.183832 3440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.183821 3442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:12.184018 3434 server_base.cc:1047] running on GCE node
I20251024 08:15:12.184218 3434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:12.184481 3434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:12.185635 3434 hybrid_clock.cc:648] HybridClock initialized: now 1761293712185622 us; error 35 us; skew 500 ppm
I20251024 08:15:12.187253 3434 webserver.cc:492] Webserver started at http://127.31.115.254:34531/ using document root <none> and password file <none>
I20251024 08:15:12.187496 3434 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:12.187548 3434 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:12.187680 3434 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:12.188616 3434 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "8516b36df09f4a5fa4612fc56bea6d8a"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.188941 3434 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "8516b36df09f4a5fa4612fc56bea6d8a"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.190471 3434 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20251024 08:15:12.191524 3448 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.191819 3434 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:12.191926 3434 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
uuid: "8516b36df09f4a5fa4612fc56bea6d8a"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.192005 3434 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:12.199757 3434 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:12.200081 3434 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:12.200205 3434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:12.204357 3434 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:45843
I20251024 08:15:12.204404 3500 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:45843 every 8 connection(s)
I20251024 08:15:12.204752 3434 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20251024 08:15:12.205444 3501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:12.207026 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 3434
I20251024 08:15:12.207145 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance
I20251024 08:15:12.208264 3501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a: Bootstrap starting.
I20251024 08:15:12.209007 3501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:12.209373 3501 log.cc:826] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:12.210089 3501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a: No bootstrap required, opened a new log
I20251024 08:15:12.211575 3501 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8516b36df09f4a5fa4612fc56bea6d8a" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45843 } }
I20251024 08:15:12.211731 3501 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:12.211763 3501 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8516b36df09f4a5fa4612fc56bea6d8a, State: Initialized, Role: FOLLOWER
I20251024 08:15:12.211874 3501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8516b36df09f4a5fa4612fc56bea6d8a" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45843 } }
I20251024 08:15:12.211957 3501 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:12.211997 3501 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:12.212044 3501 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:12.212819 3501 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8516b36df09f4a5fa4612fc56bea6d8a" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45843 } }
I20251024 08:15:12.212949 3501 leader_election.cc:304] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8516b36df09f4a5fa4612fc56bea6d8a; no voters:
I20251024 08:15:12.213114 3501 leader_election.cc:290] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:12.213207 3506 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:12.213359 3506 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [term 1 LEADER]: Becoming Leader. State: Replica: 8516b36df09f4a5fa4612fc56bea6d8a, State: Running, Role: LEADER
I20251024 08:15:12.213384 3501 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:12.213485 3506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8516b36df09f4a5fa4612fc56bea6d8a" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45843 } }
I20251024 08:15:12.213899 3508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8516b36df09f4a5fa4612fc56bea6d8a. Latest consensus state: current_term: 1 leader_uuid: "8516b36df09f4a5fa4612fc56bea6d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8516b36df09f4a5fa4612fc56bea6d8a" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45843 } } }
I20251024 08:15:12.214025 3508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:12.213876 3507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8516b36df09f4a5fa4612fc56bea6d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8516b36df09f4a5fa4612fc56bea6d8a" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 45843 } } }
I20251024 08:15:12.214347 3507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:12.214463 3515 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:12.215523 3515 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:12.217038 3515 catalog_manager.cc:1357] Generated new cluster ID: a5ecf6eeb2344cd6b47375f53c873d47
I20251024 08:15:12.217104 3515 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:12.242102 3515 catalog_manager.cc:1380] Generated new certificate authority record
I20251024 08:15:12.243000 3515 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:12.251051 3515 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 8516b36df09f4a5fa4612fc56bea6d8a: Generated new TSK 0
I20251024 08:15:12.251310 3515 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251024 08:15:12.261952 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.193:0
--local_ip_for_outbound_sockets=127.31.115.193
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45843
--builtin_ntp_servers=127.31.115.212:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20251024 08:15:12.351168 3525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:12.351401 3525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:12.351424 3525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:12.353108 3525 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20251024 08:15:12.353191 3525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:12.353224 3525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.193
I20251024 08:15:12.355199 3525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45843
--never_fsync=true
--heap_profile_path=/tmp/kudu.3525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.193
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:12.355536 3525 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:12.355821 3525 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:12.358911 3530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:12.359037 3525 server_base.cc:1047] running on GCE node
W20251024 08:15:12.358886 3531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.359297 3533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:12.359597 3525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:12.359884 3525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:12.361035 3525 hybrid_clock.cc:648] HybridClock initialized: now 1761293712361008 us; error 56 us; skew 500 ppm
I20251024 08:15:12.362607 3525 webserver.cc:492] Webserver started at http://127.31.115.193:37699/ using document root <none> and password file <none>
I20251024 08:15:12.362854 3525 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:12.362910 3525 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:12.363042 3525 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:12.363996 3525 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "0f607c09c90c4f83a9d55219583fae8b"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.364353 3525 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "0f607c09c90c4f83a9d55219583fae8b"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.365839 3525 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20251024 08:15:12.366850 3539 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.367156 3525 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:12.367266 3525 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "0f607c09c90c4f83a9d55219583fae8b"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.367349 3525 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:12.379794 3525 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:12.380100 3525 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:12.380227 3525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:12.380481 3525 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:12.380869 3525 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:12.380906 3525 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.380947 3525 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:12.380983 3525 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.389045 3525 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.193:34573
I20251024 08:15:12.389120 3652 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.193:34573 every 8 connection(s)
I20251024 08:15:12.389487 3525 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20251024 08:15:12.394316 3653 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45843
I20251024 08:15:12.394469 3653 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:12.394735 3653 heartbeater.cc:507] Master 127.31.115.254:45843 requested a full tablet report, sending...
I20251024 08:15:12.395287 3465 ts_manager.cc:194] Registered new tserver with Master: 0f607c09c90c4f83a9d55219583fae8b (127.31.115.193:34573)
I20251024 08:15:12.396021 3465 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.193:47147
I20251024 08:15:12.397858 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 3525
I20251024 08:15:12.397953 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20251024 08:15:12.399219 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.194:0
--local_ip_for_outbound_sockets=127.31.115.194
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45843
--builtin_ntp_servers=127.31.115.212:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20251024 08:15:12.482446 3656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:12.482825 3656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:12.482863 3656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:12.484490 3656 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20251024 08:15:12.484560 3656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:12.484606 3656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.194
I20251024 08:15:12.486420 3656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45843
--never_fsync=true
--heap_profile_path=/tmp/kudu.3656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.194
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:12.486760 3656 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:12.487028 3656 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:12.490041 3664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.490058 3662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.490060 3661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:12.490358 3656 server_base.cc:1047] running on GCE node
I20251024 08:15:12.490594 3656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:12.490816 3656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:12.491976 3656 hybrid_clock.cc:648] HybridClock initialized: now 1761293712491958 us; error 34 us; skew 500 ppm
I20251024 08:15:12.493537 3656 webserver.cc:492] Webserver started at http://127.31.115.194:45753/ using document root <none> and password file <none>
I20251024 08:15:12.493785 3656 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:12.493844 3656 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:12.493971 3656 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:12.495005 3656 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "857e03cf0a954782be22243407766115"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.495364 3656 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "857e03cf0a954782be22243407766115"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.496842 3656 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20251024 08:15:12.497707 3670 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.497907 3656 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:12.497995 3656 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "857e03cf0a954782be22243407766115"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.498066 3656 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:12.512485 3656 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:12.512785 3656 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:12.512909 3656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:12.513171 3656 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:12.513530 3656 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:12.513569 3656 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.513610 3656 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:12.513643 3656 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.519919 3656 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.194:40127
I20251024 08:15:12.519987 3783 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.194:40127 every 8 connection(s)
I20251024 08:15:12.520332 3656 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20251024 08:15:12.524669 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 3656
I20251024 08:15:12.524773 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20251024 08:15:12.525422 3784 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45843
I20251024 08:15:12.525553 3784 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:12.525810 3784 heartbeater.cc:507] Master 127.31.115.254:45843 requested a full tablet report, sending...
I20251024 08:15:12.526165 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.195:0
--local_ip_for_outbound_sockets=127.31.115.195
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45843
--builtin_ntp_servers=127.31.115.212:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20251024 08:15:12.526257 3464 ts_manager.cc:194] Registered new tserver with Master: 857e03cf0a954782be22243407766115 (127.31.115.194:40127)
I20251024 08:15:12.526767 3464 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.194:60139
W20251024 08:15:12.611719 3787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:12.611932 3787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:12.611954 3787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:12.613572 3787 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20251024 08:15:12.613636 3787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:12.613667 3787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.195
I20251024 08:15:12.615705 3787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45843
--never_fsync=true
--heap_profile_path=/tmp/kudu.3787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.195
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:12.615958 3787 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:12.616194 3787 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:12.619026 3793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.619029 3795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.619006 3792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:12.619318 3787 server_base.cc:1047] running on GCE node
I20251024 08:15:12.619500 3787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:12.619715 3787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:12.620872 3787 hybrid_clock.cc:648] HybridClock initialized: now 1761293712620854 us; error 33 us; skew 500 ppm
I20251024 08:15:12.622246 3787 webserver.cc:492] Webserver started at http://127.31.115.195:36171/ using document root <none> and password file <none>
I20251024 08:15:12.622475 3787 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:12.622574 3787 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:12.622730 3787 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:12.623705 3787 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "f3d57d8c5fb14143ad3b593c78c20f99"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.624042 3787 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "f3d57d8c5fb14143ad3b593c78c20f99"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.625409 3787 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20251024 08:15:12.626241 3801 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.626453 3787 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:12.626588 3787 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "f3d57d8c5fb14143ad3b593c78c20f99"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.626663 3787 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:12.636857 3787 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:12.637194 3787 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:12.637328 3787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:12.637617 3787 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:12.637979 3787 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:12.638016 3787 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.638046 3787 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:12.638057 3787 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.644630 3787 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.195:45851
I20251024 08:15:12.644690 3914 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.195:45851 every 8 connection(s)
I20251024 08:15:12.645010 3787 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20251024 08:15:12.650300 3915 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45843
I20251024 08:15:12.650429 3915 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:12.650691 3915 heartbeater.cc:507] Master 127.31.115.254:45843 requested a full tablet report, sending...
I20251024 08:15:12.651139 3464 ts_manager.cc:194] Registered new tserver with Master: f3d57d8c5fb14143ad3b593c78c20f99 (127.31.115.195:45851)
I20251024 08:15:12.651551 3464 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.195:55583
I20251024 08:15:12.651844 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 3787
I20251024 08:15:12.651953 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20251024 08:15:12.653179 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.196:0
--local_ip_for_outbound_sockets=127.31.115.196
--webserver_interface=127.31.115.196
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45843
--builtin_ntp_servers=127.31.115.212:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20251024 08:15:12.738114 3918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:12.738332 3918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:12.738354 3918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:12.740008 3918 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20251024 08:15:12.740085 3918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:12.740116 3918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.196
I20251024 08:15:12.741900 3918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.31.115.196
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45843
--never_fsync=true
--heap_profile_path=/tmp/kudu.3918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.196
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:12.742167 3918 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:12.742412 3918 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:12.745410 3923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.745410 3924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.745532 3926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:12.745858 3918 server_base.cc:1047] running on GCE node
I20251024 08:15:12.746073 3918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:12.746379 3918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:12.747612 3918 hybrid_clock.cc:648] HybridClock initialized: now 1761293712747586 us; error 59 us; skew 500 ppm
I20251024 08:15:12.749001 3918 webserver.cc:492] Webserver started at http://127.31.115.196:45431/ using document root <none> and password file <none>
I20251024 08:15:12.749219 3918 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:12.749272 3918 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:12.749374 3918 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:12.750285 3918 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "bb6e435839ca45459402f0afd42699dd"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.750699 3918 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "bb6e435839ca45459402f0afd42699dd"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.752169 3918 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:12.753014 3932 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.753199 3918 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251024 08:15:12.753290 3918 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "bb6e435839ca45459402f0afd42699dd"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.753365 3918 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:12.764923 3918 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:12.765264 3918 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:12.765388 3918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:12.765643 3918 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:12.765977 3918 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:12.766014 3918 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.766052 3918 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:12.766086 3918 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.772483 3918 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.196:44293
I20251024 08:15:12.772564 4045 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.196:44293 every 8 connection(s)
I20251024 08:15:12.772886 3918 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20251024 08:15:12.778695 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 3918
I20251024 08:15:12.778813 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20251024 08:15:12.778860 4046 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45843
I20251024 08:15:12.778985 4046 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:12.779203 4046 heartbeater.cc:507] Master 127.31.115.254:45843 requested a full tablet report, sending...
I20251024 08:15:12.779660 3465 ts_manager.cc:194] Registered new tserver with Master: bb6e435839ca45459402f0afd42699dd (127.31.115.196:44293)
I20251024 08:15:12.780150 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.197:0
--local_ip_for_outbound_sockets=127.31.115.197
--webserver_interface=127.31.115.197
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45843
--builtin_ntp_servers=127.31.115.212:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20251024 08:15:12.780179 3465 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.196:44681
W20251024 08:15:12.866918 4049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:12.867128 4049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:12.867151 4049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:12.868723 4049 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20251024 08:15:12.868788 4049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:12.868815 4049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.197
I20251024 08:15:12.870695 4049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.31.115.197
--webserver_port=0
--tserver_master_addrs=127.31.115.254:45843
--never_fsync=true
--heap_profile_path=/tmp/kudu.4049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.197
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:12.870994 4049 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:12.871300 4049 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:12.874063 4055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.874068 4054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:12.874068 4057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:12.874308 4049 server_base.cc:1047] running on GCE node
I20251024 08:15:12.874542 4049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:12.874812 4049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:12.875962 4049 hybrid_clock.cc:648] HybridClock initialized: now 1761293712875943 us; error 46 us; skew 500 ppm
I20251024 08:15:12.877323 4049 webserver.cc:492] Webserver started at http://127.31.115.197:43781/ using document root <none> and password file <none>
I20251024 08:15:12.877549 4049 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:12.877604 4049 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:12.877744 4049 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:12.878767 4049 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "e1192ace1496419dbe352b1cf1087e09"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.879119 4049 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "e1192ace1496419dbe352b1cf1087e09"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.880461 4049 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20251024 08:15:12.881315 4063 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.881546 4049 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:12.881633 4049 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "e1192ace1496419dbe352b1cf1087e09"
format_stamp: "Formatted at 2025-10-24 08:15:12 on dist-test-slave-c0j3"
I20251024 08:15:12.881708 4049 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:12.892571 4049 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:12.892863 4049 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:12.892980 4049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:12.893239 4049 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:12.893574 4049 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:12.893610 4049 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.893647 4049 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:12.893678 4049 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:12.900105 4049 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.197:37637
I20251024 08:15:12.900162 4176 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.197:37637 every 8 connection(s)
I20251024 08:15:12.900516 4049 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20251024 08:15:12.905627 4177 heartbeater.cc:344] Connected to a master server at 127.31.115.254:45843
I20251024 08:15:12.905757 4177 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:12.905961 4177 heartbeater.cc:507] Master 127.31.115.254:45843 requested a full tablet report, sending...
I20251024 08:15:12.905988 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 4049
I20251024 08:15:12.906083 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20251024 08:15:12.906405 3465 ts_manager.cc:194] Registered new tserver with Master: e1192ace1496419dbe352b1cf1087e09 (127.31.115.197:37637)
I20251024 08:15:12.906984 3465 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.197:46183
I20251024 08:15:12.907281 32207 external_mini_cluster.cc:949] 5 TS(s) registered with all masters
I20251024 08:15:12.913755 3465 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48830:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20251024 08:15:12.921895 3587 tablet_service.cc:1505] Processing CreateTablet for tablet 3b7c2b22988044bbaff48dc2d048953c (DEFAULT_TABLE table=TestTable [id=f6cccae59dae4b99ae09ac750d112334]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:12.922313 3587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b7c2b22988044bbaff48dc2d048953c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:12.922729 3718 tablet_service.cc:1505] Processing CreateTablet for tablet 3b7c2b22988044bbaff48dc2d048953c (DEFAULT_TABLE table=TestTable [id=f6cccae59dae4b99ae09ac750d112334]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:12.923044 3718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b7c2b22988044bbaff48dc2d048953c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:12.922983 4111 tablet_service.cc:1505] Processing CreateTablet for tablet 3b7c2b22988044bbaff48dc2d048953c (DEFAULT_TABLE table=TestTable [id=f6cccae59dae4b99ae09ac750d112334]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:12.923233 4111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b7c2b22988044bbaff48dc2d048953c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:12.924710 4194 tablet_bootstrap.cc:492] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b: Bootstrap starting.
I20251024 08:15:12.925356 4194 tablet_bootstrap.cc:654] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:12.925611 4194 log.cc:826] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:12.926337 4194 tablet_bootstrap.cc:492] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b: No bootstrap required, opened a new log
I20251024 08:15:12.926321 4196 tablet_bootstrap.cc:492] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09: Bootstrap starting.
I20251024 08:15:12.926410 4194 ts_tablet_manager.cc:1403] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:12.926402 4197 tablet_bootstrap.cc:492] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115: Bootstrap starting.
I20251024 08:15:12.927107 4197 tablet_bootstrap.cc:654] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:12.927107 4196 tablet_bootstrap.cc:654] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:12.927455 4197 log.cc:826] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:12.927455 4196 log.cc:826] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:12.927825 4194 raft_consensus.cc:359] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:12.928007 4194 raft_consensus.cc:385] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:12.928047 4194 raft_consensus.cc:740] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f607c09c90c4f83a9d55219583fae8b, State: Initialized, Role: FOLLOWER
I20251024 08:15:12.928170 4194 consensus_queue.cc:260] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:12.928301 4197 tablet_bootstrap.cc:492] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115: No bootstrap required, opened a new log
I20251024 08:15:12.928376 4197 ts_tablet_manager.cc:1403] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:12.928476 4196 tablet_bootstrap.cc:492] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09: No bootstrap required, opened a new log
I20251024 08:15:12.928478 4194 ts_tablet_manager.cc:1434] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251024 08:15:12.928550 4196 ts_tablet_manager.cc:1403] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:12.928614 3653 heartbeater.cc:499] Master 127.31.115.254:45843 was elected leader, sending a full tablet report...
I20251024 08:15:12.930233 4196 raft_consensus.cc:359] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:12.930215 4197 raft_consensus.cc:359] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:12.930415 4196 raft_consensus.cc:385] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:12.930415 4197 raft_consensus.cc:385] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:12.930451 4196 raft_consensus.cc:740] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1192ace1496419dbe352b1cf1087e09, State: Initialized, Role: FOLLOWER
I20251024 08:15:12.930460 4197 raft_consensus.cc:740] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857e03cf0a954782be22243407766115, State: Initialized, Role: FOLLOWER
I20251024 08:15:12.930606 4197 consensus_queue.cc:260] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:12.930606 4196 consensus_queue.cc:260] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:12.930949 4196 ts_tablet_manager.cc:1434] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251024 08:15:12.930949 4197 ts_tablet_manager.cc:1434] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251024 08:15:12.931116 4177 heartbeater.cc:499] Master 127.31.115.254:45843 was elected leader, sending a full tablet report...
I20251024 08:15:12.931131 3784 heartbeater.cc:499] Master 127.31.115.254:45843 was elected leader, sending a full tablet report...
I20251024 08:15:12.993077 4202 raft_consensus.cc:493] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251024 08:15:12.993237 4202 raft_consensus.cc:515] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:12.993567 4202 leader_election.cc:290] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f607c09c90c4f83a9d55219583fae8b (127.31.115.193:34573), 857e03cf0a954782be22243407766115 (127.31.115.194:40127)
I20251024 08:15:12.997352 3607 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b7c2b22988044bbaff48dc2d048953c" candidate_uuid: "e1192ace1496419dbe352b1cf1087e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f607c09c90c4f83a9d55219583fae8b" is_pre_election: true
I20251024 08:15:12.997604 3607 raft_consensus.cc:2468] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1192ace1496419dbe352b1cf1087e09 in term 0.
I20251024 08:15:12.997877 4065 leader_election.cc:304] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f607c09c90c4f83a9d55219583fae8b, e1192ace1496419dbe352b1cf1087e09; no voters:
I20251024 08:15:12.998039 4202 raft_consensus.cc:2804] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251024 08:15:12.998093 4202 raft_consensus.cc:493] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251024 08:15:12.998114 4202 raft_consensus.cc:3060] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:12.998245 3738 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b7c2b22988044bbaff48dc2d048953c" candidate_uuid: "e1192ace1496419dbe352b1cf1087e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857e03cf0a954782be22243407766115" is_pre_election: true
I20251024 08:15:12.998389 3738 raft_consensus.cc:2468] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1192ace1496419dbe352b1cf1087e09 in term 0.
I20251024 08:15:12.998992 4202 raft_consensus.cc:515] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:12.999130 4202 leader_election.cc:290] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [CANDIDATE]: Term 1 election: Requested vote from peers 0f607c09c90c4f83a9d55219583fae8b (127.31.115.193:34573), 857e03cf0a954782be22243407766115 (127.31.115.194:40127)
I20251024 08:15:12.999346 3607 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b7c2b22988044bbaff48dc2d048953c" candidate_uuid: "e1192ace1496419dbe352b1cf1087e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f607c09c90c4f83a9d55219583fae8b"
I20251024 08:15:12.999388 3738 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b7c2b22988044bbaff48dc2d048953c" candidate_uuid: "e1192ace1496419dbe352b1cf1087e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857e03cf0a954782be22243407766115"
I20251024 08:15:12.999454 3607 raft_consensus.cc:3060] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:12.999461 3738 raft_consensus.cc:3060] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:13.000114 3738 raft_consensus.cc:2468] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1192ace1496419dbe352b1cf1087e09 in term 1.
I20251024 08:15:13.000200 3607 raft_consensus.cc:2468] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1192ace1496419dbe352b1cf1087e09 in term 1.
I20251024 08:15:13.000316 4065 leader_election.cc:304] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 857e03cf0a954782be22243407766115, e1192ace1496419dbe352b1cf1087e09; no voters:
I20251024 08:15:13.000420 4202 raft_consensus.cc:2804] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:13.000602 4202 raft_consensus.cc:697] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [term 1 LEADER]: Becoming Leader. State: Replica: e1192ace1496419dbe352b1cf1087e09, State: Running, Role: LEADER
I20251024 08:15:13.000711 4202 consensus_queue.cc:237] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:13.001561 3465 catalog_manager.cc:5649] T 3b7c2b22988044bbaff48dc2d048953c P e1192ace1496419dbe352b1cf1087e09 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1192ace1496419dbe352b1cf1087e09 (127.31.115.197). New cstate: current_term: 1 leader_uuid: "e1192ace1496419dbe352b1cf1087e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } health_report { overall_health: UNKNOWN } } }
I20251024 08:15:13.017355 32207 external_mini_cluster.cc:949] 5 TS(s) registered with all masters
I20251024 08:15:13.017949 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 0f607c09c90c4f83a9d55219583fae8b to finish bootstrapping
W20251024 08:15:13.021078 3785 tablet.cc:2378] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251024 08:15:13.022146 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 857e03cf0a954782be22243407766115 to finish bootstrapping
I20251024 08:15:13.025851 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e1192ace1496419dbe352b1cf1087e09 to finish bootstrapping
I20251024 08:15:13.029582 32207 test_util.cc:276] Using random seed: 639384011
I20251024 08:15:13.035894 32207 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20251024 08:15:13.036088 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 4049
W20251024 08:15:13.042289 4211 negotiation.cc:337] Failed RPC negotiation. Trace:
1024 08:15:13.038327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.115.197:37637 (local address 127.0.0.1:37656)
1024 08:15:13.038400 (+ 73us) negotiation.cc:107] Waiting for socket to connect
1024 08:15:13.038412 (+ 12us) client_negotiation.cc:174] Beginning negotiation
1024 08:15:13.038460 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
1024 08:15:13.042137 (+ 3677us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.115.197:37637: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":42}
W20251024 08:15:13.042536 4210 meta_cache.cc:302] tablet 3b7c2b22988044bbaff48dc2d048953c: replica e1192ace1496419dbe352b1cf1087e09 (127.31.115.197:37637) has failed: Network error: Client connection negotiation failed: client connection to 127.31.115.197:37637: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251024 08:15:13.053483 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:13.062757 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.074296 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:13.082338 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.097615 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:13.110543 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.131736 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:13.140370 3654 tablet.cc:2378] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20251024 08:15:13.145468 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
I20251024 08:15:13.171620 3587 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:13.172302 3849 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:13.172585 3718 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20251024 08:15:13.178263 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
I20251024 08:15:13.180788 3980 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20251024 08:15:13.193310 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.233270 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:13.255412 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.298023 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:13.323184 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.371958 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:13.396947 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.452445 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:13.479454 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.538241 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:13.572288 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.641031 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
I20251024 08:15:13.652665 3915 heartbeater.cc:499] Master 127.31.115.254:45843 was elected leader, sending a full tablet report...
W20251024 08:15:13.674167 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.746192 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
I20251024 08:15:13.781172 4046 heartbeater.cc:499] Master 127.31.115.254:45843 was elected leader, sending a full tablet report...
W20251024 08:15:13.784165 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.863759 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:13.907095 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:13.991994 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:14.034329 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:14.080547 4210 meta_cache.cc:302] tablet 3b7c2b22988044bbaff48dc2d048953c: replica e1192ace1496419dbe352b1cf1087e09 (127.31.115.197:37637) has failed: Network error: Client connection negotiation failed: client connection to 127.31.115.197:37637: connect: Connection refused (error 111)
W20251024 08:15:14.128307 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:14.174744 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
I20251024 08:15:14.220259 3587 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:14.220379 3849 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:14.223750 3980 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:14.227097 3718 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20251024 08:15:14.274062 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:14.326421 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
W20251024 08:15:14.429205 3567 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39708: Illegal state: replica 0f607c09c90c4f83a9d55219583fae8b is not leader of this config: current role FOLLOWER
W20251024 08:15:14.481681 3698 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60152: Illegal state: replica 857e03cf0a954782be22243407766115 is not leader of this config: current role FOLLOWER
I20251024 08:15:14.513959 4278 raft_consensus.cc:493] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251024 08:15:14.514158 4278 raft_consensus.cc:515] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:14.514611 4278 leader_election.cc:290] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1192ace1496419dbe352b1cf1087e09 (127.31.115.197:37637), 0f607c09c90c4f83a9d55219583fae8b (127.31.115.193:34573)
W20251024 08:15:14.515126 3674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.115.197:37637: connect: Connection refused (error 111)
W20251024 08:15:14.515901 3674 leader_election.cc:336] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e1192ace1496419dbe352b1cf1087e09 (127.31.115.197:37637): Network error: Client connection negotiation failed: client connection to 127.31.115.197:37637: connect: Connection refused (error 111)
I20251024 08:15:14.518090 3607 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b7c2b22988044bbaff48dc2d048953c" candidate_uuid: "857e03cf0a954782be22243407766115" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f607c09c90c4f83a9d55219583fae8b" is_pre_election: true
I20251024 08:15:14.518204 3607 raft_consensus.cc:2468] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 857e03cf0a954782be22243407766115 in term 1.
I20251024 08:15:14.518412 3672 leader_election.cc:304] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f607c09c90c4f83a9d55219583fae8b, 857e03cf0a954782be22243407766115; no voters: e1192ace1496419dbe352b1cf1087e09
I20251024 08:15:14.518591 4278 raft_consensus.cc:2804] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251024 08:15:14.518648 4278 raft_consensus.cc:493] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251024 08:15:14.518669 4278 raft_consensus.cc:3060] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:14.519327 4278 raft_consensus.cc:515] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:14.519474 4278 leader_election.cc:290] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [CANDIDATE]: Term 2 election: Requested vote from peers e1192ace1496419dbe352b1cf1087e09 (127.31.115.197:37637), 0f607c09c90c4f83a9d55219583fae8b (127.31.115.193:34573)
I20251024 08:15:14.519697 3607 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b7c2b22988044bbaff48dc2d048953c" candidate_uuid: "857e03cf0a954782be22243407766115" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f607c09c90c4f83a9d55219583fae8b"
I20251024 08:15:14.519773 3607 raft_consensus.cc:3060] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [term 1 FOLLOWER]: Advancing to term 2
W20251024 08:15:14.520077 3674 leader_election.cc:336] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e1192ace1496419dbe352b1cf1087e09 (127.31.115.197:37637): Network error: Client connection negotiation failed: client connection to 127.31.115.197:37637: connect: Connection refused (error 111)
I20251024 08:15:14.520562 3607 raft_consensus.cc:2468] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 857e03cf0a954782be22243407766115 in term 2.
I20251024 08:15:14.520804 3672 leader_election.cc:304] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f607c09c90c4f83a9d55219583fae8b, 857e03cf0a954782be22243407766115; no voters: e1192ace1496419dbe352b1cf1087e09
I20251024 08:15:14.520928 4278 raft_consensus.cc:2804] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 2 FOLLOWER]: Leader election won for term 2
I20251024 08:15:14.521116 4278 raft_consensus.cc:697] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [term 2 LEADER]: Becoming Leader. State: Replica: 857e03cf0a954782be22243407766115, State: Running, Role: LEADER
I20251024 08:15:14.521252 4278 consensus_queue.cc:237] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } }
I20251024 08:15:14.522104 3465 catalog_manager.cc:5649] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 reported cstate change: term changed from 1 to 2, leader changed from e1192ace1496419dbe352b1cf1087e09 (127.31.115.197) to 857e03cf0a954782be22243407766115 (127.31.115.194). New cstate: current_term: 2 leader_uuid: "857e03cf0a954782be22243407766115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1192ace1496419dbe352b1cf1087e09" member_type: VOTER last_known_addr { host: "127.31.115.197" port: 37637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "857e03cf0a954782be22243407766115" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 40127 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:14.537760 3607 raft_consensus.cc:1275] T 3b7c2b22988044bbaff48dc2d048953c P 0f607c09c90c4f83a9d55219583fae8b [term 2 FOLLOWER]: Refusing update from remote peer 857e03cf0a954782be22243407766115: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20251024 08:15:14.538048 3674 consensus_peers.cc:597] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 -> Peer e1192ace1496419dbe352b1cf1087e09 (127.31.115.197:37637): Couldn't send request to peer e1192ace1496419dbe352b1cf1087e09. Status: Network error: Client connection negotiation failed: client connection to 127.31.115.197:37637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251024 08:15:14.538146 4282 consensus_queue.cc:1048] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f607c09c90c4f83a9d55219583fae8b" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 34573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:15.273175 3587 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:15.274358 3980 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:15.279126 3849 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:15.280689 3718 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251024 08:15:16.332082 3849 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:16.332270 3587 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:16.332661 3718 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251024 08:15:16.337998 3980 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20251024 08:15:16.879967 3674 consensus_peers.cc:597] T 3b7c2b22988044bbaff48dc2d048953c P 857e03cf0a954782be22243407766115 -> Peer e1192ace1496419dbe352b1cf1087e09 (127.31.115.197:37637): Couldn't send request to peer e1192ace1496419dbe352b1cf1087e09. Status: Network error: Client connection negotiation failed: client connection to 127.31.115.197:37637: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20251024 08:15:17.382001 3849 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:17.382429 3718 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251024 08:15:17.382668 3980 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:17.385547 3587 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:18.403751 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 3525
I20251024 08:15:18.410784 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 3656
I20251024 08:15:18.416709 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 3787
I20251024 08:15:18.422253 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 3918
I20251024 08:15:18.427839 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 3434
2025-10-24T08:15:18Z chronyd exiting
[ OK ] EnableKudu1097AndDownTS/MoveTabletParamTest.Test/4 (6367 ms)
[----------] 1 test from EnableKudu1097AndDownTS/MoveTabletParamTest (6367 ms total)
[----------] 1 test from ListTableCliSimpleParamTest
[ RUN ] ListTableCliSimpleParamTest.TestListTables/2
I20251024 08:15:18.449976 32207 test_util.cc:276] Using random seed: 644804402
I20251024 08:15:18.450631 32207 ts_itest-base.cc:115] Starting cluster with:
I20251024 08:15:18.450678 32207 ts_itest-base.cc:116] --------------
I20251024 08:15:18.450697 32207 ts_itest-base.cc:117] 1 tablet servers
I20251024 08:15:18.450716 32207 ts_itest-base.cc:118] 1 replicas per TS
I20251024 08:15:18.450734 32207 ts_itest-base.cc:119] --------------
2025-10-24T08:15:18Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-24T08:15:18Z Disabled control of system clock
I20251024 08:15:18.460194 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:44897
--webserver_interface=127.31.115.254
--webserver_port=0
--builtin_ntp_servers=127.31.115.212:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:44897 with env {}
W20251024 08:15:18.544410 4403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:18.544657 4403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:18.544690 4403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:18.546370 4403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:18.546459 4403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:18.546483 4403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:18.546548 4403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:18.548498 4403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:44897
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:44897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.4403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:18.548828 4403 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:18.549121 4403 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:18.552037 4409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:18.552052 4411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:18.552053 4408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:18.552043 4403 server_base.cc:1047] running on GCE node
I20251024 08:15:18.552455 4403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:18.552755 4403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:18.553920 4403 hybrid_clock.cc:648] HybridClock initialized: now 1761293718553897 us; error 47 us; skew 500 ppm
I20251024 08:15:18.555810 4403 webserver.cc:492] Webserver started at http://127.31.115.254:40417/ using document root <none> and password file <none>
I20251024 08:15:18.556131 4403 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:18.556195 4403 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:18.556350 4403 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:18.557432 4403 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3"
format_stamp: "Formatted at 2025-10-24 08:15:18 on dist-test-slave-c0j3"
I20251024 08:15:18.557773 4403 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3"
format_stamp: "Formatted at 2025-10-24 08:15:18 on dist-test-slave-c0j3"
I20251024 08:15:18.559438 4403 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20251024 08:15:18.560652 4417 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:18.560940 4403 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:18.561064 4403 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3"
format_stamp: "Formatted at 2025-10-24 08:15:18 on dist-test-slave-c0j3"
I20251024 08:15:18.561162 4403 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:18.568542 4403 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:18.568871 4403 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:18.568994 4403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:18.573096 4403 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:44897
I20251024 08:15:18.573135 4469 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:44897 every 8 connection(s)
I20251024 08:15:18.573909 4403 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20251024 08:15:18.575397 4470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:18.575978 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 4403
I20251024 08:15:18.576089 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/master-0/wal/instance
I20251024 08:15:18.579392 4470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3: Bootstrap starting.
I20251024 08:15:18.580250 4470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:18.580605 4470 log.cc:826] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:18.581359 4470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3: No bootstrap required, opened a new log
I20251024 08:15:18.583184 4470 raft_consensus.cc:359] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 44897 } }
I20251024 08:15:18.583353 4470 raft_consensus.cc:385] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:18.583376 4470 raft_consensus.cc:740] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e45407ddc24aa8ad6aac5eef9a74d3, State: Initialized, Role: FOLLOWER
I20251024 08:15:18.583518 4470 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 44897 } }
I20251024 08:15:18.583596 4470 raft_consensus.cc:399] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:18.583626 4470 raft_consensus.cc:493] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:18.583740 4470 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:18.584403 4470 raft_consensus.cc:515] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 44897 } }
I20251024 08:15:18.584529 4470 leader_election.cc:304] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2e45407ddc24aa8ad6aac5eef9a74d3; no voters:
I20251024 08:15:18.584748 4470 leader_election.cc:290] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:18.584856 4475 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:18.585049 4470 sys_catalog.cc:565] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:18.585099 4475 raft_consensus.cc:697] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [term 1 LEADER]: Becoming Leader. State: Replica: a2e45407ddc24aa8ad6aac5eef9a74d3, State: Running, Role: LEADER
I20251024 08:15:18.585283 4475 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 44897 } }
I20251024 08:15:18.585728 4477 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2e45407ddc24aa8ad6aac5eef9a74d3. Latest consensus state: current_term: 1 leader_uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 44897 } } }
I20251024 08:15:18.585744 4476 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e45407ddc24aa8ad6aac5eef9a74d3" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 44897 } } }
I20251024 08:15:18.585868 4477 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:18.585882 4476 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3 [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:18.586172 4484 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:18.586824 4484 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:18.588671 4484 catalog_manager.cc:1357] Generated new cluster ID: 62560b0de2554fa3a75814fe824d1bb3
I20251024 08:15:18.588742 4484 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:18.598466 4484 catalog_manager.cc:1380] Generated new certificate authority record
I20251024 08:15:18.599548 4484 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:18.605011 4484 catalog_manager.cc:6022] T 00000000000000000000000000000000 P a2e45407ddc24aa8ad6aac5eef9a74d3: Generated new TSK 0
I20251024 08:15:18.605273 4484 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251024 08:15:18.608590 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.193:0
--local_ip_for_outbound_sockets=127.31.115.193
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:44897
--builtin_ntp_servers=127.31.115.212:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20251024 08:15:18.737183 4494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:18.737421 4494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:18.737457 4494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:18.739230 4494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:18.739338 4494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.193
I20251024 08:15:18.741251 4494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:44897
--never_fsync=true
--heap_profile_path=/tmp/kudu.4494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.193
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:18.741593 4494 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:18.741878 4494 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:18.744987 4502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:18.744987 4500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:18.744988 4499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:18.745205 4494 server_base.cc:1047] running on GCE node
I20251024 08:15:18.745594 4494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:18.745863 4494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:18.747020 4494 hybrid_clock.cc:648] HybridClock initialized: now 1761293718747001 us; error 39 us; skew 500 ppm
I20251024 08:15:18.748417 4494 webserver.cc:492] Webserver started at http://127.31.115.193:39835/ using document root <none> and password file <none>
I20251024 08:15:18.748673 4494 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:18.748723 4494 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:18.748840 4494 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:18.749826 4494 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "9359be5e965442b3923cf78c44d37118"
format_stamp: "Formatted at 2025-10-24 08:15:18 on dist-test-slave-c0j3"
I20251024 08:15:18.750155 4494 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "9359be5e965442b3923cf78c44d37118"
format_stamp: "Formatted at 2025-10-24 08:15:18 on dist-test-slave-c0j3"
I20251024 08:15:18.751781 4494 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20251024 08:15:18.752625 4508 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:18.752830 4494 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:18.752909 4494 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "9359be5e965442b3923cf78c44d37118"
format_stamp: "Formatted at 2025-10-24 08:15:18 on dist-test-slave-c0j3"
I20251024 08:15:18.752991 4494 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:18.779371 4494 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:18.779677 4494 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:18.779794 4494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:18.780062 4494 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:18.780449 4494 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:18.780485 4494 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:18.780511 4494 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:18.780525 4494 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:18.788371 4494 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.193:42469
I20251024 08:15:18.788471 4621 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.193:42469 every 8 connection(s)
I20251024 08:15:18.788825 4494 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20251024 08:15:18.794826 4622 heartbeater.cc:344] Connected to a master server at 127.31.115.254:44897
I20251024 08:15:18.794970 4622 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:18.795209 4622 heartbeater.cc:507] Master 127.31.115.254:44897 requested a full tablet report, sending...
I20251024 08:15:18.795810 4434 ts_manager.cc:194] Registered new tserver with Master: 9359be5e965442b3923cf78c44d37118 (127.31.115.193:42469)
I20251024 08:15:18.796533 4434 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.193:60103
I20251024 08:15:18.796875 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 4494
I20251024 08:15:18.796963 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1761293700605927-32207-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20251024 08:15:18.798053 32207 external_mini_cluster.cc:949] 1 TS(s) registered with all masters
I20251024 08:15:18.805753 4434 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40372:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20251024 08:15:18.815083 4556 tablet_service.cc:1505] Processing CreateTablet for tablet e18a3fedbbd64d0ebe82e60e89d66ee8 (DEFAULT_TABLE table=TestTable [id=dc0ef1ca53ac4cf4966e800e608663c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:18.815488 4556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18a3fedbbd64d0ebe82e60e89d66ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:18.818248 4632 tablet_bootstrap.cc:492] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118: Bootstrap starting.
I20251024 08:15:18.819067 4632 tablet_bootstrap.cc:654] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:18.819401 4632 log.cc:826] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:18.820222 4632 tablet_bootstrap.cc:492] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118: No bootstrap required, opened a new log
I20251024 08:15:18.820315 4632 ts_tablet_manager.cc:1403] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251024 08:15:18.821590 4632 raft_consensus.cc:359] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9359be5e965442b3923cf78c44d37118" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 42469 } }
I20251024 08:15:18.821744 4632 raft_consensus.cc:385] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:18.821776 4632 raft_consensus.cc:740] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9359be5e965442b3923cf78c44d37118, State: Initialized, Role: FOLLOWER
I20251024 08:15:18.821885 4632 consensus_queue.cc:260] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9359be5e965442b3923cf78c44d37118" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 42469 } }
I20251024 08:15:18.821947 4632 raft_consensus.cc:399] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:18.821983 4632 raft_consensus.cc:493] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:18.822054 4632 raft_consensus.cc:3060] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:18.822803 4632 raft_consensus.cc:515] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9359be5e965442b3923cf78c44d37118" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 42469 } }
I20251024 08:15:18.822937 4632 leader_election.cc:304] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9359be5e965442b3923cf78c44d37118; no voters:
I20251024 08:15:18.823172 4632 leader_election.cc:290] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:18.823299 4634 raft_consensus.cc:2804] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:18.823475 4632 ts_tablet_manager.cc:1434] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20251024 08:15:18.823480 4634 raft_consensus.cc:697] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [term 1 LEADER]: Becoming Leader. State: Replica: 9359be5e965442b3923cf78c44d37118, State: Running, Role: LEADER
I20251024 08:15:18.823513 4622 heartbeater.cc:499] Master 127.31.115.254:44897 was elected leader, sending a full tablet report...
I20251024 08:15:18.823642 4634 consensus_queue.cc:237] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9359be5e965442b3923cf78c44d37118" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 42469 } }
I20251024 08:15:18.824575 4434 catalog_manager.cc:5649] T e18a3fedbbd64d0ebe82e60e89d66ee8 P 9359be5e965442b3923cf78c44d37118 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9359be5e965442b3923cf78c44d37118 (127.31.115.193). New cstate: current_term: 1 leader_uuid: "9359be5e965442b3923cf78c44d37118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9359be5e965442b3923cf78c44d37118" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 42469 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:18.828663 32207 external_mini_cluster.cc:949] 1 TS(s) registered with all masters
I20251024 08:15:18.829152 32207 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 9359be5e965442b3923cf78c44d37118 to finish bootstrapping
I20251024 08:15:18.991612 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 4494
I20251024 08:15:18.997499 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 4403
2025-10-24T08:15:18Z chronyd exiting
[ OK ] ListTableCliSimpleParamTest.TestListTables/2 (567 ms)
[----------] 1 test from ListTableCliSimpleParamTest (567 ms total)
[----------] 1 test from ListTableCliParamTest
[ RUN ] ListTableCliParamTest.ListTabletWithPartitionInfo/4
I20251024 08:15:19.017443 32207 test_util.cc:276] Using random seed: 645371867
[ OK ] ListTableCliParamTest.ListTabletWithPartitionInfo/4 (1 ms)
[----------] 1 test from ListTableCliParamTest (1 ms total)
[----------] 1 test from IsSecure/SecureClusterAdminCliParamTest
[ RUN ] IsSecure/SecureClusterAdminCliParamTest.TestRebuildMaster/0
2025-10-24T08:15:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-24T08:15:19Z Disabled control of system clock
I20251024 08:15:19.028080 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:39869
--webserver_interface=127.31.115.254
--webserver_port=0
--builtin_ntp_servers=127.31.115.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:39869 with env {}
W20251024 08:15:19.111793 4656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:19.112021 4656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:19.112054 4656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:19.113675 4656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:19.113744 4656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:19.113765 4656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:19.113790 4656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:19.115671 4656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:39869
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:39869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.4656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:19.116014 4656 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:19.116274 4656 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:19.118887 4664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:19.118887 4661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:19.118887 4662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:19.119138 4656 server_base.cc:1047] running on GCE node
I20251024 08:15:19.119439 4656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:19.119736 4656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:19.120899 4656 hybrid_clock.cc:648] HybridClock initialized: now 1761293719120873 us; error 50 us; skew 500 ppm
I20251024 08:15:19.122278 4656 webserver.cc:492] Webserver started at http://127.31.115.254:36955/ using document root <none> and password file <none>
I20251024 08:15:19.122581 4656 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:19.122642 4656 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:19.122784 4656 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:19.123791 4656 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/instance:
uuid: "dd9df23666284341a6841a6d4e2ab55e"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.124174 4656 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal/instance:
uuid: "dd9df23666284341a6841a6d4e2ab55e"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.125550 4656 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20251024 08:15:19.126410 4670 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:19.126662 4656 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:19.126755 4656 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
uuid: "dd9df23666284341a6841a6d4e2ab55e"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.126833 4656 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:19.138020 4656 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:19.138347 4656 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:19.138473 4656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:19.142359 4656 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:39869
I20251024 08:15:19.142405 4722 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:39869 every 8 connection(s)
I20251024 08:15:19.142799 4656 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/info.pb
I20251024 08:15:19.143414 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 4656
I20251024 08:15:19.143535 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal/instance
I20251024 08:15:19.143533 4723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:19.147292 4723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e: Bootstrap starting.
I20251024 08:15:19.148048 4723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:19.148379 4723 log.cc:826] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:19.149221 4723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e: No bootstrap required, opened a new log
I20251024 08:15:19.150454 4723 raft_consensus.cc:359] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9df23666284341a6841a6d4e2ab55e" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } }
I20251024 08:15:19.150689 4723 raft_consensus.cc:385] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:19.150717 4723 raft_consensus.cc:740] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9df23666284341a6841a6d4e2ab55e, State: Initialized, Role: FOLLOWER
I20251024 08:15:19.150811 4723 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9df23666284341a6841a6d4e2ab55e" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } }
I20251024 08:15:19.150882 4723 raft_consensus.cc:399] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:19.150918 4723 raft_consensus.cc:493] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:19.150955 4723 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:19.151551 4723 raft_consensus.cc:515] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9df23666284341a6841a6d4e2ab55e" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } }
I20251024 08:15:19.151664 4723 leader_election.cc:304] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd9df23666284341a6841a6d4e2ab55e; no voters:
I20251024 08:15:19.151863 4723 leader_election.cc:290] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:19.151942 4728 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:19.152133 4728 raft_consensus.cc:697] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [term 1 LEADER]: Becoming Leader. State: Replica: dd9df23666284341a6841a6d4e2ab55e, State: Running, Role: LEADER
I20251024 08:15:19.152154 4723 sys_catalog.cc:565] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:19.152271 4728 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9df23666284341a6841a6d4e2ab55e" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } }
I20251024 08:15:19.152653 4729 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd9df23666284341a6841a6d4e2ab55e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9df23666284341a6841a6d4e2ab55e" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } } }
I20251024 08:15:19.152680 4730 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd9df23666284341a6841a6d4e2ab55e. Latest consensus state: current_term: 1 leader_uuid: "dd9df23666284341a6841a6d4e2ab55e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9df23666284341a6841a6d4e2ab55e" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } } }
I20251024 08:15:19.152901 4730 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:19.152823 4729 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:19.153303 4737 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:19.154174 4737 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:19.156026 4737 catalog_manager.cc:1357] Generated new cluster ID: 90595127e97642eea091cf179ce4e9e4
I20251024 08:15:19.156085 4737 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:19.194031 4737 catalog_manager.cc:1380] Generated new certificate authority record
I20251024 08:15:19.194768 4737 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:19.201892 4737 catalog_manager.cc:6022] T 00000000000000000000000000000000 P dd9df23666284341a6841a6d4e2ab55e: Generated new TSK 0
I20251024 08:15:19.202121 4737 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251024 08:15:19.207743 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.193:0
--local_ip_for_outbound_sockets=127.31.115.193
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:39869
--builtin_ntp_servers=127.31.115.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20251024 08:15:19.297093 4747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:19.297312 4747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:19.297343 4747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:19.299108 4747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:19.299214 4747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.193
I20251024 08:15:19.301074 4747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.115.193
--webserver_port=0
--tserver_master_addrs=127.31.115.254:39869
--never_fsync=true
--heap_profile_path=/tmp/kudu.4747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.193
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:19.301414 4747 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:19.301694 4747 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:19.304869 4755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:19.304901 4753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:19.304885 4752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:19.305809 4747 server_base.cc:1047] running on GCE node
I20251024 08:15:19.306110 4747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:19.306476 4747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:19.307725 4747 hybrid_clock.cc:648] HybridClock initialized: now 1761293719307701 us; error 42 us; skew 500 ppm
I20251024 08:15:19.309715 4747 webserver.cc:492] Webserver started at http://127.31.115.193:41065/ using document root <none> and password file <none>
I20251024 08:15:19.310029 4747 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:19.310096 4747 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:19.310252 4747 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:19.311705 4747 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/instance:
uuid: "1a369a95c99f4ae8b267f7a4870cb90c"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.312244 4747 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal/instance:
uuid: "1a369a95c99f4ae8b267f7a4870cb90c"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.314193 4747 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251024 08:15:19.315394 4761 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:19.315719 4747 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:19.315817 4747 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal
uuid: "1a369a95c99f4ae8b267f7a4870cb90c"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.315898 4747 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:19.333882 4747 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:19.334260 4747 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:19.334378 4747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:19.334736 4747 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:19.335165 4747 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:19.335202 4747 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:19.335230 4747 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:19.335263 4747 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:19.343348 4747 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.193:46023
I20251024 08:15:19.343643 4874 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.193:46023 every 8 connection(s)
I20251024 08:15:19.343942 4747 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/info.pb
I20251024 08:15:19.351650 4875 heartbeater.cc:344] Connected to a master server at 127.31.115.254:39869
I20251024 08:15:19.351815 4875 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:19.352087 4875 heartbeater.cc:507] Master 127.31.115.254:39869 requested a full tablet report, sending...
I20251024 08:15:19.352809 4687 ts_manager.cc:194] Registered new tserver with Master: 1a369a95c99f4ae8b267f7a4870cb90c (127.31.115.193:46023)
I20251024 08:15:19.353838 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 4747
I20251024 08:15:19.353916 4687 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.193:39909
I20251024 08:15:19.353974 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal/instance
I20251024 08:15:19.355429 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.194:0
--local_ip_for_outbound_sockets=127.31.115.194
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:39869
--builtin_ntp_servers=127.31.115.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20251024 08:15:19.446784 4878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:19.446995 4878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:19.447019 4878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:19.448621 4878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:19.448700 4878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.194
I20251024 08:15:19.450492 4878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.115.194
--webserver_port=0
--tserver_master_addrs=127.31.115.254:39869
--never_fsync=true
--heap_profile_path=/tmp/kudu.4878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.194
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:19.450804 4878 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:19.451035 4878 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:19.453984 4884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:19.453985 4886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:19.454090 4878 server_base.cc:1047] running on GCE node
W20251024 08:15:19.453984 4883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:19.454401 4878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:19.454749 4878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:19.455909 4878 hybrid_clock.cc:648] HybridClock initialized: now 1761293719455899 us; error 42 us; skew 500 ppm
I20251024 08:15:19.457422 4878 webserver.cc:492] Webserver started at http://127.31.115.194:34875/ using document root <none> and password file <none>
I20251024 08:15:19.457665 4878 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:19.457713 4878 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:19.457842 4878 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:19.458871 4878 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/instance:
uuid: "b19c51b4871047adb47025403bedf5a0"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.459218 4878 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal/instance:
uuid: "b19c51b4871047adb47025403bedf5a0"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.460606 4878 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:19.461496 4892 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:19.461694 4878 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:19.461776 4878 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal
uuid: "b19c51b4871047adb47025403bedf5a0"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.461854 4878 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:19.483443 4878 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:19.483734 4878 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:19.483875 4878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:19.484107 4878 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:19.484491 4878 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:19.484529 4878 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:19.484553 4878 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:19.484581 4878 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:19.490945 4878 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.194:41907
I20251024 08:15:19.491005 5005 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.194:41907 every 8 connection(s)
I20251024 08:15:19.491364 4878 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/info.pb
I20251024 08:15:19.496593 5006 heartbeater.cc:344] Connected to a master server at 127.31.115.254:39869
I20251024 08:15:19.496713 5006 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:19.496924 5006 heartbeater.cc:507] Master 127.31.115.254:39869 requested a full tablet report, sending...
I20251024 08:15:19.497349 4687 ts_manager.cc:194] Registered new tserver with Master: b19c51b4871047adb47025403bedf5a0 (127.31.115.194:41907)
I20251024 08:15:19.497751 4687 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.194:52013
I20251024 08:15:19.501552 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 4878
I20251024 08:15:19.501657 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal/instance
I20251024 08:15:19.503166 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.195:0
--local_ip_for_outbound_sockets=127.31.115.195
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:39869
--builtin_ntp_servers=127.31.115.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20251024 08:15:19.591250 5009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:19.591460 5009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:19.591482 5009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:19.593021 5009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:19.593096 5009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.195
I20251024 08:15:19.594883 5009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.115.195
--webserver_port=0
--tserver_master_addrs=127.31.115.254:39869
--never_fsync=true
--heap_profile_path=/tmp/kudu.5009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.195
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:19.595144 5009 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:19.595388 5009 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:19.598302 5015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:19.598304 5017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:19.598304 5014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:19.598573 5009 server_base.cc:1047] running on GCE node
I20251024 08:15:19.598829 5009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:19.599087 5009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:19.600256 5009 hybrid_clock.cc:648] HybridClock initialized: now 1761293719600237 us; error 39 us; skew 500 ppm
I20251024 08:15:19.601619 5009 webserver.cc:492] Webserver started at http://127.31.115.195:42331/ using document root <none> and password file <none>
I20251024 08:15:19.601864 5009 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:19.601919 5009 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:19.602061 5009 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:19.603082 5009 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/instance:
uuid: "a6a07721ce214d48ac8e9ec0b817e0e4"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.603440 5009 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal/instance:
uuid: "a6a07721ce214d48ac8e9ec0b817e0e4"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.604791 5009 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20251024 08:15:19.605674 5023 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:19.605912 5009 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:19.606004 5009 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal
uuid: "a6a07721ce214d48ac8e9ec0b817e0e4"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:19.606081 5009 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:19.614076 5009 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:19.614384 5009 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:19.614558 5009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:19.614786 5009 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:19.615113 5009 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251024 08:15:19.615162 5009 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:19.615199 5009 ts_tablet_manager.cc:616] Registered 0 tablets
I20251024 08:15:19.615231 5009 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:19.621564 5009 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.195:36853
I20251024 08:15:19.621614 5136 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.195:36853 every 8 connection(s)
I20251024 08:15:19.621974 5009 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/info.pb
I20251024 08:15:19.626922 5137 heartbeater.cc:344] Connected to a master server at 127.31.115.254:39869
I20251024 08:15:19.627075 5137 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:19.627341 5137 heartbeater.cc:507] Master 127.31.115.254:39869 requested a full tablet report, sending...
I20251024 08:15:19.627779 4687 ts_manager.cc:194] Registered new tserver with Master: a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853)
I20251024 08:15:19.628180 4687 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.195:46889
I20251024 08:15:19.628594 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 5009
I20251024 08:15:19.628705 32207 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal/instance
I20251024 08:15:19.629854 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:19.636669 32207 test_util.cc:276] Using random seed: 645991098
I20251024 08:15:19.643442 4687 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58396:
name: "pre_rebuild"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251024 08:15:19.643733 4687 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table pre_rebuild in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251024 08:15:19.650787 4940 tablet_service.cc:1505] Processing CreateTablet for tablet 1a2276b43809495490512b6699fc5dd7 (DEFAULT_TABLE table=pre_rebuild [id=c5f80a7179a045bbb8b2be84505c1a32]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:19.651149 4940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a2276b43809495490512b6699fc5dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:19.651582 4809 tablet_service.cc:1505] Processing CreateTablet for tablet 1a2276b43809495490512b6699fc5dd7 (DEFAULT_TABLE table=pre_rebuild [id=c5f80a7179a045bbb8b2be84505c1a32]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:19.651873 4809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a2276b43809495490512b6699fc5dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:19.654242 5156 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Bootstrap starting.
I20251024 08:15:19.654692 5157 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Bootstrap starting.
I20251024 08:15:19.655175 5156 tablet_bootstrap.cc:654] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:19.655175 5071 tablet_service.cc:1505] Processing CreateTablet for tablet 1a2276b43809495490512b6699fc5dd7 (DEFAULT_TABLE table=pre_rebuild [id=c5f80a7179a045bbb8b2be84505c1a32]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:19.655400 5157 tablet_bootstrap.cc:654] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:19.655488 5071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a2276b43809495490512b6699fc5dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:19.655519 5156 log.cc:826] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:19.655684 5157 log.cc:826] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:19.656315 5156 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: No bootstrap required, opened a new log
I20251024 08:15:19.656369 5156 ts_tablet_manager.cc:1403] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251024 08:15:19.656412 5157 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: No bootstrap required, opened a new log
I20251024 08:15:19.656478 5157 ts_tablet_manager.cc:1403] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:19.657532 5156 raft_consensus.cc:359] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:19.657684 5156 raft_consensus.cc:385] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:19.657725 5156 raft_consensus.cc:740] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b19c51b4871047adb47025403bedf5a0, State: Initialized, Role: FOLLOWER
I20251024 08:15:19.657833 5156 consensus_queue.cc:260] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:19.658150 5156 ts_tablet_manager.cc:1434] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251024 08:15:19.658314 5006 heartbeater.cc:499] Master 127.31.115.254:39869 was elected leader, sending a full tablet report...
I20251024 08:15:19.658743 5157 raft_consensus.cc:359] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:19.658883 5157 raft_consensus.cc:385] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:19.658914 5157 raft_consensus.cc:740] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a369a95c99f4ae8b267f7a4870cb90c, State: Initialized, Role: FOLLOWER
I20251024 08:15:19.659039 5157 consensus_queue.cc:260] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:19.659288 5157 ts_tablet_manager.cc:1434] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251024 08:15:19.659350 4875 heartbeater.cc:499] Master 127.31.115.254:39869 was elected leader, sending a full tablet report...
I20251024 08:15:19.659500 5161 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Bootstrap starting.
I20251024 08:15:19.660086 5161 tablet_bootstrap.cc:654] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:19.660383 5161 log.cc:826] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:19.661077 5161 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: No bootstrap required, opened a new log
I20251024 08:15:19.661146 5161 ts_tablet_manager.cc:1403] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:19.662497 5161 raft_consensus.cc:359] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:19.662701 5161 raft_consensus.cc:385] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:19.662734 5161 raft_consensus.cc:740] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6a07721ce214d48ac8e9ec0b817e0e4, State: Initialized, Role: FOLLOWER
I20251024 08:15:19.662853 5161 consensus_queue.cc:260] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:19.663128 5161 ts_tablet_manager.cc:1434] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251024 08:15:19.663322 5137 heartbeater.cc:499] Master 127.31.115.254:39869 was elected leader, sending a full tablet report...
I20251024 08:15:19.673313 5160 raft_consensus.cc:493] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251024 08:15:19.673501 5160 raft_consensus.cc:515] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:19.673903 5160 leader_election.cc:290] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853), 1a369a95c99f4ae8b267f7a4870cb90c (127.31.115.193:46023)
I20251024 08:15:19.678433 5091 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a2276b43809495490512b6699fc5dd7" candidate_uuid: "b19c51b4871047adb47025403bedf5a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" is_pre_election: true
I20251024 08:15:19.678463 4829 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a2276b43809495490512b6699fc5dd7" candidate_uuid: "b19c51b4871047adb47025403bedf5a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" is_pre_election: true
I20251024 08:15:19.678752 4829 raft_consensus.cc:2468] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b19c51b4871047adb47025403bedf5a0 in term 0.
I20251024 08:15:19.678728 5091 raft_consensus.cc:2468] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b19c51b4871047adb47025403bedf5a0 in term 0.
I20251024 08:15:19.679222 4895 leader_election.cc:304] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6a07721ce214d48ac8e9ec0b817e0e4, b19c51b4871047adb47025403bedf5a0; no voters:
I20251024 08:15:19.679536 5160 raft_consensus.cc:2804] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251024 08:15:19.679653 5160 raft_consensus.cc:493] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251024 08:15:19.679697 5160 raft_consensus.cc:3060] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:19.680636 5160 raft_consensus.cc:515] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:19.680979 5160 leader_election.cc:290] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [CANDIDATE]: Term 1 election: Requested vote from peers a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853), 1a369a95c99f4ae8b267f7a4870cb90c (127.31.115.193:46023)
I20251024 08:15:19.681226 5091 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a2276b43809495490512b6699fc5dd7" candidate_uuid: "b19c51b4871047adb47025403bedf5a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4"
I20251024 08:15:19.681340 5091 raft_consensus.cc:3060] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:19.681336 4829 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a2276b43809495490512b6699fc5dd7" candidate_uuid: "b19c51b4871047adb47025403bedf5a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a369a95c99f4ae8b267f7a4870cb90c"
I20251024 08:15:19.681448 4829 raft_consensus.cc:3060] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:19.682152 4829 raft_consensus.cc:2468] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b19c51b4871047adb47025403bedf5a0 in term 1.
I20251024 08:15:19.682165 5091 raft_consensus.cc:2468] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b19c51b4871047adb47025403bedf5a0 in term 1.
I20251024 08:15:19.682425 4894 leader_election.cc:304] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a369a95c99f4ae8b267f7a4870cb90c, b19c51b4871047adb47025403bedf5a0; no voters:
I20251024 08:15:19.682590 5160 raft_consensus.cc:2804] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:19.682799 5160 raft_consensus.cc:697] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 1 LEADER]: Becoming Leader. State: Replica: b19c51b4871047adb47025403bedf5a0, State: Running, Role: LEADER
I20251024 08:15:19.682914 5160 consensus_queue.cc:237] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:19.683904 4687 catalog_manager.cc:5649] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to b19c51b4871047adb47025403bedf5a0 (127.31.115.194). New cstate: current_term: 1 leader_uuid: "b19c51b4871047adb47025403bedf5a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } health_report { overall_health: UNKNOWN } } }
I20251024 08:15:19.703544 4829 raft_consensus.cc:1275] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Refusing update from remote peer b19c51b4871047adb47025403bedf5a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251024 08:15:19.703718 5091 raft_consensus.cc:1275] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 1 FOLLOWER]: Refusing update from remote peer b19c51b4871047adb47025403bedf5a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251024 08:15:19.703975 5166 consensus_queue.cc:1048] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:19.704154 5160 consensus_queue.cc:1048] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:19.708472 5174 mvcc.cc:204] Tried to move back new op lower bound from 7214259075901915136 to 7214259075822116864. Current Snapshot: MvccSnapshot[applied={T|T < 7214259075901915136}]
I20251024 08:15:19.708475 5173 mvcc.cc:204] Tried to move back new op lower bound from 7214259075904704512 to 7214259075822116864. Current Snapshot: MvccSnapshot[applied={T|T < 7214259075901915136}]
I20251024 08:15:19.712400 5176 mvcc.cc:204] Tried to move back new op lower bound from 7214259075904704512 to 7214259075822116864. Current Snapshot: MvccSnapshot[applied={T|T < 7214259075901915136}]
I20251024 08:15:19.897547 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 4656
W20251024 08:15:19.995540 5194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:19.995795 5194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:19.997409 5194 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20251024 08:15:20.019488 5194 minidump.cc:252] Setting minidump size limit to 20M
I20251024 08:15:20.019886 5194 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:20.020123 5194 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:20.022850 5216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:20.022881 5215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:20.023056 5194 server_base.cc:1047] running on GCE node
W20251024 08:15:20.022850 5218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:20.023475 5194 hybrid_clock.cc:584] initializing the hybrid clock with 'system' time source
I20251024 08:15:20.023591 5194 hybrid_clock.cc:648] HybridClock initialized: now 1761293720023590 us; error 33715 us; skew 500 ppm
I20251024 08:15:20.024711 5194 webserver.cc:492] Webserver started at http://0.0.0.0:35759/ using document root <none> and password file <none>
I20251024 08:15:20.024971 5194 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:20.025044 5194 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:20.025161 5194 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251024 08:15:20.026422 5194 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/instance:
uuid: "a3989aebc1dc47248a39df46e15ccece"
format_stamp: "Formatted at 2025-10-24 08:15:20 on dist-test-slave-c0j3"
I20251024 08:15:20.027097 5194 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal/instance:
uuid: "a3989aebc1dc47248a39df46e15ccece"
format_stamp: "Formatted at 2025-10-24 08:15:20 on dist-test-slave-c0j3"
I20251024 08:15:20.028860 5194 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20251024 08:15:20.029806 5223 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:20.029995 5194 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:20.030097 5194 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
uuid: "a3989aebc1dc47248a39df46e15ccece"
format_stamp: "Formatted at 2025-10-24 08:15:20 on dist-test-slave-c0j3"
I20251024 08:15:20.030179 5194 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:20.099373 5194 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:20.099705 5194 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:20.099817 5194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:20.100306 5194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:20.102780 5194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: Bootstrap starting.
I20251024 08:15:20.103408 5194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:20.103981 5194 log.cc:826] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:20.104694 5194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: No bootstrap required, opened a new log
I20251024 08:15:20.105998 5194 raft_consensus.cc:359] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER }
I20251024 08:15:20.106132 5194 raft_consensus.cc:385] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:20.106153 5194 raft_consensus.cc:740] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3989aebc1dc47248a39df46e15ccece, State: Initialized, Role: FOLLOWER
I20251024 08:15:20.106269 5194 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER }
I20251024 08:15:20.106334 5194 raft_consensus.cc:399] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:20.106359 5194 raft_consensus.cc:493] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:20.106417 5194 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:20.107064 5194 raft_consensus.cc:515] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER }
I20251024 08:15:20.107195 5194 leader_election.cc:304] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3989aebc1dc47248a39df46e15ccece; no voters:
I20251024 08:15:20.107400 5194 leader_election.cc:290] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [CANDIDATE]: Term 1 election: Requested vote from peers
I20251024 08:15:20.107484 5230 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:20.107654 5230 raft_consensus.cc:697] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 1 LEADER]: Becoming Leader. State: Replica: a3989aebc1dc47248a39df46e15ccece, State: Running, Role: LEADER
I20251024 08:15:20.107791 5230 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER }
I20251024 08:15:20.108182 5231 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3989aebc1dc47248a39df46e15ccece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER } }
I20251024 08:15:20.108309 5231 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:20.108222 5232 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3989aebc1dc47248a39df46e15ccece. Latest consensus state: current_term: 1 leader_uuid: "a3989aebc1dc47248a39df46e15ccece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER } }
I20251024 08:15:20.108428 5232 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:20.109726 5194 tablet_replica.cc:333] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: stopping tablet replica
I20251024 08:15:20.109818 5194 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 1 LEADER]: Raft consensus shutting down.
I20251024 08:15:20.109886 5194 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 1 FOLLOWER]: Raft consensus is shut down!
I20251024 08:15:20.110246 5194 master.cc:561] Master@0.0.0.0:7051 shutting down...
W20251024 08:15:20.110404 5194 acceptor_pool.cc:194] Could not shut down acceptor socket on 0.0.0.0:7051: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20251024 08:15:20.121548 5194 master.cc:583] Master@0.0.0.0:7051 shutdown complete.
I20251024 08:15:20.128978 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 4747
I20251024 08:15:20.134205 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 4878
I20251024 08:15:20.140565 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 5009
I20251024 08:15:20.146109 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:39869
--webserver_interface=127.31.115.254
--webserver_port=36955
--builtin_ntp_servers=127.31.115.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.115.254:39869 with env {}
W20251024 08:15:20.241410 5236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:20.241648 5236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:20.241683 5236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:20.243427 5236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251024 08:15:20.243508 5236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:20.243533 5236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251024 08:15:20.243551 5236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251024 08:15:20.245386 5236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.115.254:39869
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.115.254:39869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.115.254
--webserver_port=36955
--never_fsync=true
--heap_profile_path=/tmp/kudu.5236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:20.245695 5236 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:20.245952 5236 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:20.248975 5244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:20.248975 5242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:20.249122 5241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:20.249490 5236 server_base.cc:1047] running on GCE node
I20251024 08:15:20.249917 5236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:20.250289 5236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:20.251497 5236 hybrid_clock.cc:648] HybridClock initialized: now 1761293720251465 us; error 73 us; skew 500 ppm
I20251024 08:15:20.253226 5236 webserver.cc:492] Webserver started at http://127.31.115.254:36955/ using document root <none> and password file <none>
I20251024 08:15:20.253481 5236 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:20.253557 5236 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:20.255200 5236 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20251024 08:15:20.256479 5250 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:20.256803 5236 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:20.256927 5236 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
uuid: "a3989aebc1dc47248a39df46e15ccece"
format_stamp: "Formatted at 2025-10-24 08:15:20 on dist-test-slave-c0j3"
I20251024 08:15:20.257246 5236 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:20.275821 5236 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:20.276289 5236 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:20.276439 5236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:20.281924 5236 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.254:39869
I20251024 08:15:20.281987 5302 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.254:39869 every 8 connection(s)
I20251024 08:15:20.282398 5236 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/info.pb
I20251024 08:15:20.282923 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 5236
I20251024 08:15:20.283496 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.193:46023
--local_ip_for_outbound_sockets=127.31.115.193
--tserver_master_addrs=127.31.115.254:39869
--webserver_port=41065
--webserver_interface=127.31.115.193
--builtin_ntp_servers=127.31.115.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20251024 08:15:20.283893 5303 sys_catalog.cc:263] Verifying existing consensus state
I20251024 08:15:20.286262 5303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: Bootstrap starting.
I20251024 08:15:20.288497 5303 log.cc:826] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:20.295522 5303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=2 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:20.295917 5303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: Bootstrap complete.
I20251024 08:15:20.297628 5303 raft_consensus.cc:359] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } }
I20251024 08:15:20.297808 5303 raft_consensus.cc:740] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3989aebc1dc47248a39df46e15ccece, State: Initialized, Role: FOLLOWER
I20251024 08:15:20.297972 5303 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } }
I20251024 08:15:20.298060 5303 raft_consensus.cc:399] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251024 08:15:20.298115 5303 raft_consensus.cc:493] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251024 08:15:20.298180 5303 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:20.298992 5303 raft_consensus.cc:515] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } }
I20251024 08:15:20.299129 5303 leader_election.cc:304] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3989aebc1dc47248a39df46e15ccece; no voters:
I20251024 08:15:20.299335 5303 leader_election.cc:290] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [CANDIDATE]: Term 2 election: Requested vote from peers
I20251024 08:15:20.299487 5307 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 2 FOLLOWER]: Leader election won for term 2
I20251024 08:15:20.299695 5303 sys_catalog.cc:565] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [sys.catalog]: configured and running, proceeding with master startup.
I20251024 08:15:20.299739 5307 raft_consensus.cc:697] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [term 2 LEADER]: Becoming Leader. State: Replica: a3989aebc1dc47248a39df46e15ccece, State: Running, Role: LEADER
I20251024 08:15:20.299880 5307 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } }
I20251024 08:15:20.300328 5308 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a3989aebc1dc47248a39df46e15ccece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } } }
I20251024 08:15:20.300453 5308 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:20.300725 5309 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3989aebc1dc47248a39df46e15ccece. Latest consensus state: current_term: 2 leader_uuid: "a3989aebc1dc47248a39df46e15ccece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3989aebc1dc47248a39df46e15ccece" member_type: VOTER last_known_addr { host: "127.31.115.254" port: 39869 } } }
I20251024 08:15:20.300836 5309 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece [sys.catalog]: This master's current role is: LEADER
I20251024 08:15:20.301155 5315 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251024 08:15:20.301967 5315 catalog_manager.cc:679] Loaded metadata for table pre_rebuild [id=eae3d7c1de9d4a548a63f165a2aed7c8]
I20251024 08:15:20.302234 5315 tablet_loader.cc:96] loaded metadata for tablet 1a2276b43809495490512b6699fc5dd7 (table pre_rebuild [id=eae3d7c1de9d4a548a63f165a2aed7c8])
I20251024 08:15:20.302306 5315 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251024 08:15:20.304150 5315 catalog_manager.cc:1357] Generated new cluster ID: 5b17117cb4504a599f64aaa0d583ee32
I20251024 08:15:20.304229 5315 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251024 08:15:20.312963 5315 catalog_manager.cc:1380] Generated new certificate authority record
I20251024 08:15:20.313768 5315 catalog_manager.cc:1514] Loading token signing keys...
I20251024 08:15:20.325018 5315 catalog_manager.cc:6022] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: Generated new TSK 0
I20251024 08:15:20.325268 5315 catalog_manager.cc:1524] Initializing in-progress tserver states...
W20251024 08:15:20.390975 5305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:20.391203 5305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:20.391225 5305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:20.392977 5305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:20.393069 5305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.193
I20251024 08:15:20.395020 5305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.193:46023
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.115.193
--webserver_port=41065
--tserver_master_addrs=127.31.115.254:39869
--never_fsync=true
--heap_profile_path=/tmp/kudu.5305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.193
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:20.395344 5305 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:20.395635 5305 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:20.398551 5331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:20.398566 5330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:20.398729 5305 server_base.cc:1047] running on GCE node
W20251024 08:15:20.398846 5333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:20.399075 5305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:20.399335 5305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:20.400493 5305 hybrid_clock.cc:648] HybridClock initialized: now 1761293720400466 us; error 50 us; skew 500 ppm
I20251024 08:15:20.401994 5305 webserver.cc:492] Webserver started at http://127.31.115.193:41065/ using document root <none> and password file <none>
I20251024 08:15:20.402251 5305 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:20.402326 5305 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:20.403786 5305 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:20.404609 5339 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:20.404853 5305 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:20.404938 5305 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal
uuid: "1a369a95c99f4ae8b267f7a4870cb90c"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:20.405249 5305 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:20.415370 5305 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:20.415681 5305 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:20.415812 5305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:20.416047 5305 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:20.416587 5346 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20251024 08:15:20.417529 5305 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251024 08:15:20.417587 5305 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:20.417616 5305 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251024 08:15:20.418213 5305 ts_tablet_manager.cc:616] Registered 1 tablets
I20251024 08:15:20.418259 5305 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:20.418310 5346 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Bootstrap starting.
I20251024 08:15:20.426375 5305 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.193:46023
I20251024 08:15:20.426641 5453 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.193:46023 every 8 connection(s)
I20251024 08:15:20.426942 5305 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/info.pb
I20251024 08:15:20.429955 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 5305
I20251024 08:15:20.430315 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.194:41907
--local_ip_for_outbound_sockets=127.31.115.194
--tserver_master_addrs=127.31.115.254:39869
--webserver_port=34875
--webserver_interface=127.31.115.194
--builtin_ntp_servers=127.31.115.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20251024 08:15:20.437808 5346 log.cc:826] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:20.439082 5454 heartbeater.cc:344] Connected to a master server at 127.31.115.254:39869
I20251024 08:15:20.439189 5454 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:20.439452 5454 heartbeater.cc:507] Master 127.31.115.254:39869 requested a full tablet report, sending...
I20251024 08:15:20.440586 5267 ts_manager.cc:194] Registered new tserver with Master: 1a369a95c99f4ae8b267f7a4870cb90c (127.31.115.193:46023)
I20251024 08:15:20.441761 5267 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.193:46127
I20251024 08:15:20.471369 5346 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Bootstrap replayed 1/1 log segments. Stats: ops{read=211 overwritten=0 applied=211 ignored=0} inserts{seen=10500 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:20.471779 5346 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Bootstrap complete.
I20251024 08:15:20.472762 5346 ts_tablet_manager.cc:1403] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Time spent bootstrapping tablet: real 0.055s user 0.039s sys 0.012s
I20251024 08:15:20.473703 5346 raft_consensus.cc:359] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:20.473922 5346 raft_consensus.cc:740] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a369a95c99f4ae8b267f7a4870cb90c, State: Initialized, Role: FOLLOWER
I20251024 08:15:20.474073 5346 consensus_queue.cc:260] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 211, Last appended: 1.211, Last appended by leader: 211, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:20.474355 5346 ts_tablet_manager.cc:1434] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20251024 08:15:20.474556 5454 heartbeater.cc:499] Master 127.31.115.254:39869 was elected leader, sending a full tablet report...
W20251024 08:15:20.544808 5458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:20.545071 5458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:20.545096 5458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:20.546985 5458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:20.547070 5458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.194
I20251024 08:15:20.548930 5458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.194:41907
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.115.194
--webserver_port=34875
--tserver_master_addrs=127.31.115.254:39869
--never_fsync=true
--heap_profile_path=/tmp/kudu.5458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.194
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:20.549256 5458 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:20.549546 5458 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:20.552858 5465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:20.552896 5468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:20.552876 5466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:20.553238 5458 server_base.cc:1047] running on GCE node
I20251024 08:15:20.553442 5458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:20.553714 5458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:20.554867 5458 hybrid_clock.cc:648] HybridClock initialized: now 1761293720554848 us; error 46 us; skew 500 ppm
I20251024 08:15:20.556483 5458 webserver.cc:492] Webserver started at http://127.31.115.194:34875/ using document root <none> and password file <none>
I20251024 08:15:20.556713 5458 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:20.556771 5458 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:20.558195 5458 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:20.559113 5474 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:20.559490 5458 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:20.559602 5458 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal
uuid: "b19c51b4871047adb47025403bedf5a0"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:20.559916 5458 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:20.579071 5458 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:20.579387 5458 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:20.579525 5458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:20.579757 5458 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:20.580243 5481 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20251024 08:15:20.581236 5458 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251024 08:15:20.581296 5458 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:20.581339 5458 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251024 08:15:20.581966 5458 ts_tablet_manager.cc:616] Registered 1 tablets
I20251024 08:15:20.582005 5458 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:20.582083 5481 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Bootstrap starting.
I20251024 08:15:20.591032 5458 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.194:41907
I20251024 08:15:20.591425 5588 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.194:41907 every 8 connection(s)
I20251024 08:15:20.591614 5458 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/info.pb
I20251024 08:15:20.595777 5481 log.cc:826] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:20.598057 5589 heartbeater.cc:344] Connected to a master server at 127.31.115.254:39869
I20251024 08:15:20.598186 5589 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:20.598417 5589 heartbeater.cc:507] Master 127.31.115.254:39869 requested a full tablet report, sending...
I20251024 08:15:20.599129 5267 ts_manager.cc:194] Registered new tserver with Master: b19c51b4871047adb47025403bedf5a0 (127.31.115.194:41907)
I20251024 08:15:20.599820 5267 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.194:38619
I20251024 08:15:20.599844 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 5458
I20251024 08:15:20.600198 32207 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasksTSBtL/build/release/bin/kudu
/tmp/dist-test-tasksTSBtL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.115.195:36853
--local_ip_for_outbound_sockets=127.31.115.195
--tserver_master_addrs=127.31.115.254:39869
--webserver_port=42331
--webserver_interface=127.31.115.195
--builtin_ntp_servers=127.31.115.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20251024 08:15:20.629483 5481 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Bootstrap replayed 1/1 log segments. Stats: ops{read=211 overwritten=0 applied=211 ignored=0} inserts{seen=10500 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:20.629935 5481 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Bootstrap complete.
I20251024 08:15:20.630991 5481 ts_tablet_manager.cc:1403] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Time spent bootstrapping tablet: real 0.049s user 0.025s sys 0.023s
I20251024 08:15:20.632122 5481 raft_consensus.cc:359] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:20.632339 5481 raft_consensus.cc:740] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b19c51b4871047adb47025403bedf5a0, State: Initialized, Role: FOLLOWER
I20251024 08:15:20.632462 5481 consensus_queue.cc:260] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 211, Last appended: 1.211, Last appended by leader: 211, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:20.632799 5481 ts_tablet_manager.cc:1434] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251024 08:15:20.632841 5589 heartbeater.cc:499] Master 127.31.115.254:39869 was elected leader, sending a full tablet report...
W20251024 08:15:20.699376 5593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251024 08:15:20.699622 5593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251024 08:15:20.699645 5593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251024 08:15:20.701285 5593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251024 08:15:20.701357 5593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.115.195
I20251024 08:15:20.703099 5593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.115.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.115.195:36853
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.115.195
--webserver_port=42331
--tserver_master_addrs=127.31.115.254:39869
--never_fsync=true
--heap_profile_path=/tmp/kudu.5593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.115.195
--log_dir=/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8d75a6f6cef564a7be95d0a4dbcc58aa159edfc7
build type RELEASE
built by None at 24 Oct 2025 07:43:14 UTC on 5fd53c4cbb9d
build id 8691
I20251024 08:15:20.703397 5593 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251024 08:15:20.703971 5593 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251024 08:15:20.707027 5599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:20.707031 5602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251024 08:15:20.707027 5600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251024 08:15:20.707317 5593 server_base.cc:1047] running on GCE node
I20251024 08:15:20.707506 5593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251024 08:15:20.707803 5593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251024 08:15:20.708967 5593 hybrid_clock.cc:648] HybridClock initialized: now 1761293720708943 us; error 43 us; skew 500 ppm
I20251024 08:15:20.710615 5593 webserver.cc:492] Webserver started at http://127.31.115.195:42331/ using document root <none> and password file <none>
I20251024 08:15:20.710879 5593 fs_manager.cc:362] Metadata directory not provided
I20251024 08:15:20.710954 5593 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251024 08:15:20.712498 5593 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20251024 08:15:20.713423 5608 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251024 08:15:20.713703 5593 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251024 08:15:20.713804 5593 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal
uuid: "a6a07721ce214d48ac8e9ec0b817e0e4"
format_stamp: "Formatted at 2025-10-24 08:15:19 on dist-test-slave-c0j3"
I20251024 08:15:20.714115 5593 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251024 08:15:20.732066 5593 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251024 08:15:20.732378 5593 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251024 08:15:20.732514 5593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251024 08:15:20.732731 5593 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251024 08:15:20.733201 5615 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20251024 08:15:20.734181 5593 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251024 08:15:20.734238 5593 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:20.734277 5593 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251024 08:15:20.734917 5593 ts_tablet_manager.cc:616] Registered 1 tablets
I20251024 08:15:20.734956 5593 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251024 08:15:20.735001 5615 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Bootstrap starting.
I20251024 08:15:20.741209 5593 rpc_server.cc:307] RPC server started. Bound to: 127.31.115.195:36853
I20251024 08:15:20.741284 5722 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.115.195:36853 every 8 connection(s)
I20251024 08:15:20.741639 5593 server_base.cc:1179] Dumped server information to /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/info.pb
I20251024 08:15:20.747826 5723 heartbeater.cc:344] Connected to a master server at 127.31.115.254:39869
I20251024 08:15:20.747999 5723 heartbeater.cc:461] Registering TS with master...
I20251024 08:15:20.748320 5723 heartbeater.cc:507] Master 127.31.115.254:39869 requested a full tablet report, sending...
I20251024 08:15:20.749042 5267 ts_manager.cc:194] Registered new tserver with Master: a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853)
I20251024 08:15:20.749724 5267 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.115.195:45469
I20251024 08:15:20.751035 5615 log.cc:826] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Log is configured to *not* fsync() on all Append() calls
I20251024 08:15:20.751268 32207 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasksTSBtL/build/release/bin/kudu as pid 5593
I20251024 08:15:20.755846 32207 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251024 08:15:20.785805 5615 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=211 overwritten=0 applied=211 ignored=0} inserts{seen=10500 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251024 08:15:20.786230 5615 tablet_bootstrap.cc:492] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Bootstrap complete.
I20251024 08:15:20.787182 5615 ts_tablet_manager.cc:1403] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Time spent bootstrapping tablet: real 0.052s user 0.047s sys 0.004s
I20251024 08:15:20.788133 5615 raft_consensus.cc:359] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:20.788354 5615 raft_consensus.cc:740] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6a07721ce214d48ac8e9ec0b817e0e4, State: Initialized, Role: FOLLOWER
I20251024 08:15:20.788462 5615 consensus_queue.cc:260] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 211, Last appended: 1.211, Last appended by leader: 211, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:20.788758 5615 ts_tablet_manager.cc:1434] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20251024 08:15:20.788873 5723 heartbeater.cc:499] Master 127.31.115.254:39869 was elected leader, sending a full tablet report...
I20251024 08:15:21.928565 5734 raft_consensus.cc:493] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251024 08:15:21.928763 5734 raft_consensus.cc:515] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:21.929185 5734 leader_election.cc:290] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853), b19c51b4871047adb47025403bedf5a0 (127.31.115.194:41907)
I20251024 08:15:21.933308 5543 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a2276b43809495490512b6699fc5dd7" candidate_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" candidate_term: 2 candidate_status { last_received { term: 1 index: 211 } } ignore_live_leader: false dest_uuid: "b19c51b4871047adb47025403bedf5a0" is_pre_election: true
I20251024 08:15:21.933311 5677 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a2276b43809495490512b6699fc5dd7" candidate_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" candidate_term: 2 candidate_status { last_received { term: 1 index: 211 } } ignore_live_leader: false dest_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" is_pre_election: true
I20251024 08:15:21.933552 5543 raft_consensus.cc:2468] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a369a95c99f4ae8b267f7a4870cb90c in term 1.
I20251024 08:15:21.933552 5677 raft_consensus.cc:2468] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a369a95c99f4ae8b267f7a4870cb90c in term 1.
I20251024 08:15:21.933928 5342 leader_election.cc:304] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a369a95c99f4ae8b267f7a4870cb90c, a6a07721ce214d48ac8e9ec0b817e0e4; no voters:
I20251024 08:15:21.934190 5734 raft_consensus.cc:2804] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251024 08:15:21.934300 5734 raft_consensus.cc:493] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251024 08:15:21.934325 5734 raft_consensus.cc:3060] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:21.935936 5734 raft_consensus.cc:515] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:21.936120 5734 leader_election.cc:290] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [CANDIDATE]: Term 2 election: Requested vote from peers a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853), b19c51b4871047adb47025403bedf5a0 (127.31.115.194:41907)
I20251024 08:15:21.936468 5543 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a2276b43809495490512b6699fc5dd7" candidate_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" candidate_term: 2 candidate_status { last_received { term: 1 index: 211 } } ignore_live_leader: false dest_uuid: "b19c51b4871047adb47025403bedf5a0"
I20251024 08:15:21.936480 5677 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a2276b43809495490512b6699fc5dd7" candidate_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" candidate_term: 2 candidate_status { last_received { term: 1 index: 211 } } ignore_live_leader: false dest_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4"
I20251024 08:15:21.936635 5677 raft_consensus.cc:3060] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:21.936635 5543 raft_consensus.cc:3060] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 1 FOLLOWER]: Advancing to term 2
I20251024 08:15:21.938108 5543 raft_consensus.cc:2468] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a369a95c99f4ae8b267f7a4870cb90c in term 2.
I20251024 08:15:21.938108 5677 raft_consensus.cc:2468] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a369a95c99f4ae8b267f7a4870cb90c in term 2.
I20251024 08:15:21.938575 5340 leader_election.cc:304] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a369a95c99f4ae8b267f7a4870cb90c, a6a07721ce214d48ac8e9ec0b817e0e4, b19c51b4871047adb47025403bedf5a0; no voters:
I20251024 08:15:21.938766 5734 raft_consensus.cc:2804] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 2 FOLLOWER]: Leader election won for term 2
I20251024 08:15:21.938930 5734 raft_consensus.cc:697] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 2 LEADER]: Becoming Leader. State: Replica: 1a369a95c99f4ae8b267f7a4870cb90c, State: Running, Role: LEADER
I20251024 08:15:21.939045 5734 consensus_queue.cc:237] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 211, Committed index: 211, Last appended: 1.211, Last appended by leader: 211, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:21.939990 5267 catalog_manager.cc:5649] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c reported cstate change: term changed from 0 to 2, leader changed from <none> to 1a369a95c99f4ae8b267f7a4870cb90c (127.31.115.193), VOTER 1a369a95c99f4ae8b267f7a4870cb90c (127.31.115.193) added, VOTER a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195) added, VOTER b19c51b4871047adb47025403bedf5a0 (127.31.115.194) added. New cstate: current_term: 2 leader_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } health_report { overall_health: HEALTHY } } }
W20251024 08:15:22.276492 32207 scanner-internal.cc:458] Time spent opening tablet: real 1.514s user 0.000s sys 0.001s
I20251024 08:15:22.322925 5523 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:22.322966 5388 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251024 08:15:22.331192 5657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
a3989aebc1dc47248a39df46e15ccece | 127.31.115.254:39869 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.31.115.212:46345 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
1a369a95c99f4ae8b267f7a4870cb90c | 127.31.115.193:46023 | HEALTHY | <none> | 1 | 0
a6a07721ce214d48ac8e9ec0b817e0e4 | 127.31.115.195:36853 | HEALTHY | <none> | 0 | 0
b19c51b4871047adb47025403bedf5a0 | 127.31.115.194:41907 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.31.115.193 | experimental | 127.31.115.193:46023
local_ip_for_outbound_sockets | 127.31.115.194 | experimental | 127.31.115.194:41907
local_ip_for_outbound_sockets | 127.31.115.195 | experimental | 127.31.115.195:36853
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/info.pb | hidden | 127.31.115.193:46023
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/info.pb | hidden | 127.31.115.194:41907
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/info.pb | hidden | 127.31.115.195:36853
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.31.115.212:46345 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1a2276b43809495490512b6699fc5dd7 of table 'pre_rebuild' is conflicted: 2 replicas' active configs disagree with the leader master's
a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853): RUNNING
b19c51b4871047adb47025403bedf5a0 (127.31.115.194:41907): RUNNING
1a369a95c99f4ae8b267f7a4870cb90c (127.31.115.193:46023): RUNNING [LEADER]
All reported replicas are:
A = a6a07721ce214d48ac8e9ec0b817e0e4
B = b19c51b4871047adb47025403bedf5a0
C = 1a369a95c99f4ae8b267f7a4870cb90c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 2 | -1 | Yes
B | A B C | 2 | -1 | Yes
C | A B C* | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+--------------------+---------------+---------+------------+------------------+-------------
pre_rebuild | 1 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20251024 08:15:22.340255 32207 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251024 08:15:22.506232 5543 raft_consensus.cc:1275] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 2 FOLLOWER]: Refusing update from remote peer 1a369a95c99f4ae8b267f7a4870cb90c: Log matching property violated. Preceding OpId in replica: term: 1 index: 211. Preceding OpId from leader: term: 2 index: 212. (index mismatch)
I20251024 08:15:22.506636 5784 consensus_queue.cc:1048] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 212, Last known committed idx: 211, Time since last communication: 0.000s
I20251024 08:15:22.508673 5677 raft_consensus.cc:1275] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 2 FOLLOWER]: Refusing update from remote peer 1a369a95c99f4ae8b267f7a4870cb90c: Log matching property violated. Preceding OpId in replica: term: 1 index: 211. Preceding OpId from leader: term: 2 index: 212. (index mismatch)
I20251024 08:15:22.508999 5787 consensus_queue.cc:1048] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 212, Last known committed idx: 211, Time since last communication: 0.000s
I20251024 08:15:22.513020 5408 consensus_queue.cc:237] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 212, Committed index: 212, Last appended: 2.212, Last appended by leader: 211, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 213 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.513625 5543 raft_consensus.cc:1275] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 2 FOLLOWER]: Refusing update from remote peer 1a369a95c99f4ae8b267f7a4870cb90c: Log matching property violated. Preceding OpId in replica: term: 2 index: 212. Preceding OpId from leader: term: 2 index: 213. (index mismatch)
I20251024 08:15:22.513844 5787 consensus_queue.cc:1048] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 213, Last known committed idx: 212, Time since last communication: 0.000s
I20251024 08:15:22.515128 5787 raft_consensus.cc:2955] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 2 LEADER]: Committing config change with OpId 2.213: config changed from index -1 to 213, VOTER a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195) evicted. New config: { opid_index: 213 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } } }
I20251024 08:15:22.515281 5543 raft_consensus.cc:2955] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 2 FOLLOWER]: Committing config change with OpId 2.213: config changed from index -1 to 213, VOTER a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195) evicted. New config: { opid_index: 213 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } } }
I20251024 08:15:22.516690 5252 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 1a2276b43809495490512b6699fc5dd7 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251024 08:15:22.516705 5267 catalog_manager.cc:5649] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c reported cstate change: config changed from index -1 to 213, VOTER a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195) evicted. New cstate: current_term: 2 leader_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" committed_config { opid_index: 213 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:22.518198 5408 consensus_queue.cc:237] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 213, Committed index: 213, Last appended: 2.213, Last appended by leader: 211, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.518496 5784 raft_consensus.cc:2955] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 2 LEADER]: Committing config change with OpId 2.214: config changed from index 213 to 214, VOTER b19c51b4871047adb47025403bedf5a0 (127.31.115.194) evicted. New config: { opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } } }
I20251024 08:15:22.520359 5266 catalog_manager.cc:5649] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c reported cstate change: config changed from index 213 to 214, VOTER b19c51b4871047adb47025403bedf5a0 (127.31.115.194) evicted. New cstate: current_term: 2 leader_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" committed_config { opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:22.520359 5252 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 1a2276b43809495490512b6699fc5dd7 with cas_config_opid_index 213: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251024 08:15:22.524267 5657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:22.524371 5388 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251024 08:15:22.524653 5657 tablet_service.cc:1552] Processing DeleteTablet for tablet 1a2276b43809495490512b6699fc5dd7 with delete_type TABLET_DATA_TOMBSTONED (TS a6a07721ce214d48ac8e9ec0b817e0e4 not found in new config with opid_index 213) from {username='slave'} at 127.0.0.1:35110
I20251024 08:15:22.524950 5810 tablet_replica.cc:333] stopping tablet replica
I20251024 08:15:22.525060 5810 raft_consensus.cc:2243] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251024 08:15:22.525127 5810 raft_consensus.cc:2272] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251024 08:15:22.525799 5523 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:22.526997 5810 ts_tablet_manager.cc:1916] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251024 08:15:22.528896 5810 ts_tablet_manager.cc:1929] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.212
I20251024 08:15:22.529035 5810 log.cc:1199] T 1a2276b43809495490512b6699fc5dd7 P a6a07721ce214d48ac8e9ec0b817e0e4: Deleting WAL directory at /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/wal/wals/1a2276b43809495490512b6699fc5dd7
I20251024 08:15:22.529704 5253 catalog_manager.cc:4985] TS a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853): tablet 1a2276b43809495490512b6699fc5dd7 (table pre_rebuild [id=eae3d7c1de9d4a548a63f165a2aed7c8]) successfully deleted
I20251024 08:15:22.533386 5523 tablet_service.cc:1552] Processing DeleteTablet for tablet 1a2276b43809495490512b6699fc5dd7 with delete_type TABLET_DATA_TOMBSTONED (TS b19c51b4871047adb47025403bedf5a0 not found in new config with opid_index 214) from {username='slave'} at 127.0.0.1:59806
I20251024 08:15:22.533747 5811 tablet_replica.cc:333] stopping tablet replica
I20251024 08:15:22.533849 5811 raft_consensus.cc:2243] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251024 08:15:22.533937 5811 raft_consensus.cc:2272] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251024 08:15:22.534337 5811 ts_tablet_manager.cc:1916] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
a3989aebc1dc47248a39df46e15ccece | 127.31.115.254:39869 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.31.115.212:46345 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
1a369a95c99f4ae8b267f7a4870cb90c | 127.31.115.193:46023 | HEALTHY | <none> | 1 | 0
a6a07721ce214d48ac8e9ec0b817e0e4 | 127.31.115.195:36853 | HEALTHY | <none> | 0 | 0
b19c51b4871047adb47025403bedf5a0 | 127.31.115.194:41907 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.31.115.193 | experimental | 127.31.115.193:46023
local_ip_for_outbound_sockets | 127.31.115.194 | experimental | 127.31.115.194:41907
local_ip_for_outbound_sockets | 127.31.115.195 | experimental | 127.31.115.195:36853
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/info.pb | hidden | 127.31.115.193:46023
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/info.pb | hidden | 127.31.115.194:41907
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/info.pb | hidden | 127.31.115.195:36853
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.31.115.212:46345 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1a2276b43809495490512b6699fc5dd7 of table 'pre_rebuild' is conflicted: 2 replicas' active configs disagree with the leader master's
a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853): RUNNING
b19c51b4871047adb47025403bedf5a0 (127.31.115.194:41907): RUNNING
1a369a95c99f4ae8b267f7a4870cb90c (127.31.115.193:46023): RUNNING [LEADER]
All reported replicas are:
A = a6a07721ce214d48ac8e9ec0b817e0e4
B = b19c51b4871047adb47025403bedf5a0
C = 1a369a95c99f4ae8b267f7a4870cb90c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 2 | -1 | Yes
B | B C* | 2 | 213 | Yes
C | C* | 2 | 214 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+--------------------+---------------+---------+------------+------------------+-------------
pre_rebuild | 1 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20251024 08:15:22.536392 32207 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251024 08:15:22.536674 5811 ts_tablet_manager.cc:1929] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.213
I20251024 08:15:22.536765 5811 log.cc:1199] T 1a2276b43809495490512b6699fc5dd7 P b19c51b4871047adb47025403bedf5a0: Deleting WAL directory at /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/wal/wals/1a2276b43809495490512b6699fc5dd7
I20251024 08:15:22.537323 5251 catalog_manager.cc:4985] TS b19c51b4871047adb47025403bedf5a0 (127.31.115.194:41907): tablet 1a2276b43809495490512b6699fc5dd7 (table pre_rebuild [id=eae3d7c1de9d4a548a63f165a2aed7c8]) successfully deleted
I20251024 08:15:22.790426 5657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:22.790575 5523 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:22.790846 5388 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
a3989aebc1dc47248a39df46e15ccece | 127.31.115.254:39869 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.31.115.212:46345 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
1a369a95c99f4ae8b267f7a4870cb90c | 127.31.115.193:46023 | HEALTHY | <none> | 1 | 0
a6a07721ce214d48ac8e9ec0b817e0e4 | 127.31.115.195:36853 | HEALTHY | <none> | 0 | 0
b19c51b4871047adb47025403bedf5a0 | 127.31.115.194:41907 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.31.115.193 | experimental | 127.31.115.193:46023
local_ip_for_outbound_sockets | 127.31.115.194 | experimental | 127.31.115.194:41907
local_ip_for_outbound_sockets | 127.31.115.195 | experimental | 127.31.115.195:36853
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/info.pb | hidden | 127.31.115.193:46023
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/info.pb | hidden | 127.31.115.194:41907
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/info.pb | hidden | 127.31.115.195:36853
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.31.115.212:46345 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
pre_rebuild | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 0
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20251024 08:15:22.800704 32207 log_verifier.cc:126] Checking tablet 1a2276b43809495490512b6699fc5dd7
I20251024 08:15:22.814761 32207 log_verifier.cc:177] Verified matching terms for 214 ops in tablet 1a2276b43809495490512b6699fc5dd7
I20251024 08:15:22.815207 5267 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58410:
table { table_name: "pre_rebuild" } modify_external_catalogs: true
I20251024 08:15:22.815313 5267 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58410:
table { table_name: "pre_rebuild" } modify_external_catalogs: true
I20251024 08:15:22.816156 5267 catalog_manager.cc:5936] T 00000000000000000000000000000000 P a3989aebc1dc47248a39df46e15ccece: Sending DeleteTablet for 1 replicas of tablet 1a2276b43809495490512b6699fc5dd7
I20251024 08:15:22.816567 32207 test_util.cc:276] Using random seed: 649170997
I20251024 08:15:22.816548 5388 tablet_service.cc:1552] Processing DeleteTablet for tablet 1a2276b43809495490512b6699fc5dd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-24 08:15:22 UTC) from {username='slave'} at 127.0.0.1:37920
I20251024 08:15:22.816800 5835 tablet_replica.cc:333] stopping tablet replica
I20251024 08:15:22.816915 5835 raft_consensus.cc:2243] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 2 LEADER]: Raft consensus shutting down.
I20251024 08:15:22.816989 5835 raft_consensus.cc:2272] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c [term 2 FOLLOWER]: Raft consensus is shut down!
I20251024 08:15:22.818409 5835 ts_tablet_manager.cc:1916] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251024 08:15:22.820438 5835 ts_tablet_manager.cc:1929] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.214
I20251024 08:15:22.820575 5835 log.cc:1199] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Deleting WAL directory at /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/wal/wals/1a2276b43809495490512b6699fc5dd7
I20251024 08:15:22.820919 5835 ts_tablet_manager.cc:1950] T 1a2276b43809495490512b6699fc5dd7 P 1a369a95c99f4ae8b267f7a4870cb90c: Deleting consensus metadata
I20251024 08:15:22.821529 5252 catalog_manager.cc:4985] TS 1a369a95c99f4ae8b267f7a4870cb90c (127.31.115.193:46023): tablet 1a2276b43809495490512b6699fc5dd7 (table pre_rebuild [id=eae3d7c1de9d4a548a63f165a2aed7c8]) successfully deleted
I20251024 08:15:22.822980 5267 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58494:
name: "post_rebuild"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251024 08:15:22.823422 5267 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table post_rebuild in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251024 08:15:22.825981 5523 tablet_service.cc:1505] Processing CreateTablet for tablet 35e9474c7bba402381d49e86bd6baa33 (DEFAULT_TABLE table=post_rebuild [id=c398fa9784a645bbb742acd230fddbdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:22.826020 5657 tablet_service.cc:1505] Processing CreateTablet for tablet 35e9474c7bba402381d49e86bd6baa33 (DEFAULT_TABLE table=post_rebuild [id=c398fa9784a645bbb742acd230fddbdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:22.826056 5388 tablet_service.cc:1505] Processing CreateTablet for tablet 35e9474c7bba402381d49e86bd6baa33 (DEFAULT_TABLE table=post_rebuild [id=c398fa9784a645bbb742acd230fddbdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251024 08:15:22.826253 5657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e9474c7bba402381d49e86bd6baa33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:22.826254 5523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e9474c7bba402381d49e86bd6baa33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:22.826262 5388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e9474c7bba402381d49e86bd6baa33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251024 08:15:22.828006 5843 tablet_bootstrap.cc:492] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4: Bootstrap starting.
I20251024 08:15:22.828087 5841 tablet_bootstrap.cc:492] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c: Bootstrap starting.
I20251024 08:15:22.828154 5842 tablet_bootstrap.cc:492] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0: Bootstrap starting.
I20251024 08:15:22.828434 5843 tablet_bootstrap.cc:654] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:22.828509 5842 tablet_bootstrap.cc:654] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:22.828518 5841 tablet_bootstrap.cc:654] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c: Neither blocks nor log segments found. Creating new log.
I20251024 08:15:22.829032 5843 tablet_bootstrap.cc:492] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4: No bootstrap required, opened a new log
I20251024 08:15:22.829108 5843 ts_tablet_manager.cc:1403] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:22.829196 5842 tablet_bootstrap.cc:492] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0: No bootstrap required, opened a new log
I20251024 08:15:22.829253 5842 ts_tablet_manager.cc:1403] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:22.829246 5843 raft_consensus.cc:359] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.829331 5843 raft_consensus.cc:385] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:22.829377 5843 raft_consensus.cc:740] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6a07721ce214d48ac8e9ec0b817e0e4, State: Initialized, Role: FOLLOWER
I20251024 08:15:22.829403 5842 raft_consensus.cc:359] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.829444 5843 consensus_queue.cc:260] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.829481 5842 raft_consensus.cc:385] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:22.829548 5842 raft_consensus.cc:740] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b19c51b4871047adb47025403bedf5a0, State: Initialized, Role: FOLLOWER
I20251024 08:15:22.829548 5843 ts_tablet_manager.cc:1434] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251024 08:15:22.829634 5842 consensus_queue.cc:260] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.829911 5842 ts_tablet_manager.cc:1434] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251024 08:15:22.829943 5841 tablet_bootstrap.cc:492] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c: No bootstrap required, opened a new log
I20251024 08:15:22.829993 5841 ts_tablet_manager.cc:1403] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251024 08:15:22.830168 5841 raft_consensus.cc:359] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.830286 5841 raft_consensus.cc:385] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251024 08:15:22.830338 5841 raft_consensus.cc:740] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a369a95c99f4ae8b267f7a4870cb90c, State: Initialized, Role: FOLLOWER
I20251024 08:15:22.830403 5841 consensus_queue.cc:260] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.830618 5841 ts_tablet_manager.cc:1434] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20251024 08:15:22.844465 5590 tablet.cc:2378] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251024 08:15:22.896111 5787 raft_consensus.cc:493] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251024 08:15:22.896286 5787 raft_consensus.cc:515] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.896533 5787 leader_election.cc:290] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b19c51b4871047adb47025403bedf5a0 (127.31.115.194:41907), a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853)
I20251024 08:15:22.896853 5543 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e9474c7bba402381d49e86bd6baa33" candidate_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b19c51b4871047adb47025403bedf5a0" is_pre_election: true
I20251024 08:15:22.896831 5677 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e9474c7bba402381d49e86bd6baa33" candidate_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" is_pre_election: true
I20251024 08:15:22.896957 5677 raft_consensus.cc:2468] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a369a95c99f4ae8b267f7a4870cb90c in term 0.
I20251024 08:15:22.896961 5543 raft_consensus.cc:2468] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a369a95c99f4ae8b267f7a4870cb90c in term 0.
I20251024 08:15:22.897156 5342 leader_election.cc:304] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a369a95c99f4ae8b267f7a4870cb90c, a6a07721ce214d48ac8e9ec0b817e0e4; no voters:
I20251024 08:15:22.897300 5787 raft_consensus.cc:2804] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251024 08:15:22.897372 5787 raft_consensus.cc:493] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251024 08:15:22.897388 5787 raft_consensus.cc:3060] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:22.898053 5787 raft_consensus.cc:515] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.898216 5787 leader_election.cc:290] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [CANDIDATE]: Term 1 election: Requested vote from peers b19c51b4871047adb47025403bedf5a0 (127.31.115.194:41907), a6a07721ce214d48ac8e9ec0b817e0e4 (127.31.115.195:36853)
I20251024 08:15:22.898567 5677 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e9474c7bba402381d49e86bd6baa33" candidate_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4"
I20251024 08:15:22.898567 5543 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e9474c7bba402381d49e86bd6baa33" candidate_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b19c51b4871047adb47025403bedf5a0"
I20251024 08:15:22.898698 5677 raft_consensus.cc:3060] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:22.898701 5543 raft_consensus.cc:3060] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0 [term 0 FOLLOWER]: Advancing to term 1
I20251024 08:15:22.899516 5543 raft_consensus.cc:2468] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a369a95c99f4ae8b267f7a4870cb90c in term 1.
I20251024 08:15:22.899518 5677 raft_consensus.cc:2468] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a369a95c99f4ae8b267f7a4870cb90c in term 1.
I20251024 08:15:22.899813 5340 leader_election.cc:304] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a369a95c99f4ae8b267f7a4870cb90c, b19c51b4871047adb47025403bedf5a0; no voters:
I20251024 08:15:22.899968 5787 raft_consensus.cc:2804] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 FOLLOWER]: Leader election won for term 1
I20251024 08:15:22.900048 5787 raft_consensus.cc:697] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [term 1 LEADER]: Becoming Leader. State: Replica: 1a369a95c99f4ae8b267f7a4870cb90c, State: Running, Role: LEADER
I20251024 08:15:22.900113 5787 consensus_queue.cc:237] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } } peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } }
I20251024 08:15:22.900820 5266 catalog_manager.cc:5649] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a369a95c99f4ae8b267f7a4870cb90c (127.31.115.193). New cstate: current_term: 1 leader_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a369a95c99f4ae8b267f7a4870cb90c" member_type: VOTER last_known_addr { host: "127.31.115.193" port: 46023 } health_report { overall_health: HEALTHY } } }
I20251024 08:15:22.911545 5543 raft_consensus.cc:1275] T 35e9474c7bba402381d49e86bd6baa33 P b19c51b4871047adb47025403bedf5a0 [term 1 FOLLOWER]: Refusing update from remote peer 1a369a95c99f4ae8b267f7a4870cb90c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251024 08:15:22.911545 5677 raft_consensus.cc:1275] T 35e9474c7bba402381d49e86bd6baa33 P a6a07721ce214d48ac8e9ec0b817e0e4 [term 1 FOLLOWER]: Refusing update from remote peer 1a369a95c99f4ae8b267f7a4870cb90c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251024 08:15:22.911826 5787 consensus_queue.cc:1048] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b19c51b4871047adb47025403bedf5a0" member_type: VOTER last_known_addr { host: "127.31.115.194" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:22.911931 5784 consensus_queue.cc:1048] T 35e9474c7bba402381d49e86bd6baa33 P 1a369a95c99f4ae8b267f7a4870cb90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6a07721ce214d48ac8e9ec0b817e0e4" member_type: VOTER last_known_addr { host: "127.31.115.195" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251024 08:15:22.914567 5798 mvcc.cc:204] Tried to move back new op lower bound from 7214259089043574784 to 7214259088999219200. Current Snapshot: MvccSnapshot[applied={T|T < 7214259089043574784}]
I20251024 08:15:22.914669 5790 mvcc.cc:204] Tried to move back new op lower bound from 7214259089043574784 to 7214259088999219200. Current Snapshot: MvccSnapshot[applied={T|T < 7214259089043574784 or (T in {7214259089043574784})}]
I20251024 08:15:23.116411 5523 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:23.116571 5657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251024 08:15:23.116745 5388 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
a3989aebc1dc47248a39df46e15ccece | 127.31.115.254:39869 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.31.115.212:46345 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
1a369a95c99f4ae8b267f7a4870cb90c | 127.31.115.193:46023 | HEALTHY | <none> | 1 | 0
a6a07721ce214d48ac8e9ec0b817e0e4 | 127.31.115.195:36853 | HEALTHY | <none> | 0 | 0
b19c51b4871047adb47025403bedf5a0 | 127.31.115.194:41907 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.31.115.193 | experimental | 127.31.115.193:46023
local_ip_for_outbound_sockets | 127.31.115.194 | experimental | 127.31.115.194:41907
local_ip_for_outbound_sockets | 127.31.115.195 | experimental | 127.31.115.195:36853
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-0/data/info.pb | hidden | 127.31.115.193:46023
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-1/data/info.pb | hidden | 127.31.115.194:41907
server_dump_info_path | /tmp/dist-test-tasksTSBtL/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1761293700605927-32207-0/minicluster-data/ts-2/data/info.pb | hidden | 127.31.115.195:36853
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.31.115.212:46345 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
--------------+----+---------+---------------+---------+------------+------------------+-------------
post_rebuild | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20251024 08:15:23.127017 32207 log_verifier.cc:126] Checking tablet 1a2276b43809495490512b6699fc5dd7
I20251024 08:15:23.127130 32207 log_verifier.cc:177] Verified matching terms for 0 ops in tablet 1a2276b43809495490512b6699fc5dd7
I20251024 08:15:23.127178 32207 log_verifier.cc:126] Checking tablet 35e9474c7bba402381d49e86bd6baa33
I20251024 08:15:23.164228 32207 log_verifier.cc:177] Verified matching terms for 205 ops in tablet 35e9474c7bba402381d49e86bd6baa33
I20251024 08:15:23.166075 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 5305
I20251024 08:15:23.173048 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 5458
I20251024 08:15:23.178609 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 5593
I20251024 08:15:23.184165 32207 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasksTSBtL/build/release/bin/kudu with pid 5236
2025-10-24T08:15:23Z chronyd exiting
[ OK ] IsSecure/SecureClusterAdminCliParamTest.TestRebuildMaster/0 (4187 ms)
[----------] 1 test from IsSecure/SecureClusterAdminCliParamTest (4187 ms total)
[----------] Global test environment tear-down
[==========] 9 tests from 5 test suites ran. (22594 ms total)
[ PASSED ] 8 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] AdminCliTest.TestRebuildTables
1 FAILED TEST
I20251024 08:15:23.207450 32207 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/client/meta_cache.cc:302: suppressed but not reported on 2 messages since previous log ~9 seconds ago